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A MASS MENTAL TEST FOR USE WITH KINDER- 
GARTEN AND FIRST GRADE CHILDREN 


By Ciara Harrison Town, Ph. D., 


Every year finds a greater number of public schools at- 
tempting to classify their pupils according to mental ability, 
and to measure their mental ability by means of mental tests. 
As an adequate individual mental examination of all children 
in a large school system is impossible without a large force 
of mental examiners the mass test has come into quite general 
use. 

There are a number of mass tests adapted for use with 
children of the third and more advanced grades, but it has 
been found difficult to devise tests that can be used success- 
fully with groups of kindergarten and first grade children. 
As the early grades are those in which mental classification 
is most urgently needed, it is much to be desired that if mass 
testing in these grades is possible, a satisfactory test be devised. 

During the winter of 1919- 20. the writer was confronted 
with the problem of picking out the atypical children in certain 
kindergarten and first grades in the schools of Kansas City, 
Missouri, for the purpose of placing them in classes for in- 
dividual training. Desiring to use for this purpose a mass 
test an attempt was made to devise one which would bring 
into play mental activities which Binet found developed in 
young school children, and which would accomplish this with- 
out requiring a verbal or written response from the child. 

The writer fully realized the difficulties involved in testing 
young children in groups, realized the flightiness of attention 
which renders concerted group work almost impossible, the 
strong individualistic tendencies, which disappear after several 
years of class room experience, but which at this period often 
lead a child to a response quite other than that indicated, the 
tendency to look to see how others are responding rather than 
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to depend upon self, which may entirely defeat the purpose of 
the test, and the short fatigue period, which makes maximum 
effort throughout a long test series impossible. In spite of 
this keen realization of the innate difficulties of the situation 
which made the achievement of a satisfactory test most prob- 
lematic, the writer thought the demand for such a test justified 
the expenditure of much time and thought in an effort to 
devise it. 

Bearing in mind the difficult factors in the practical situa- 
tion, an effort was made to (1) secure test material which 
would immediately arouse interest and pleasure, (2) avoid 
confusion of impression by presenting each problem in isolation 
and (3) guard against wandering of the attention by using a 
series of tests each of which is quickly solvable and through 
which the examiner may personally lead the class without 
delay. It was decided to adapt, wherever possible, the material 
of the Binet-Simon Intelligence Scale to mass methods. As 
verbal and written responses were both ruled out, the first 
by the mass nature of the test and the second by the inability 
of the children to write, the “cross out” method, so success- 
fully used by Dr. Pressey and others, was linked with the 
Binet problems and a “ Picture Game” evolved. 

The “ Picture Game” consists of a sixteen page picture 
book, five inches by seven inches, which is here reproduced. 
Almost all children love pictures and when one proposes a 
game to a class and distributes little books full of pictures 
with which to play it, there is usually a glad and happy re- 
sponse,—interest and pleasure are at once aroused. The suc- 
cess of the test depends largely upon whether this pleasurable 
interest can be sustained throughout the performance. This 
the writer tried to insure by a series of short and varied tests 
which a class can be put through in from ten to fifteen minutes. 

The game consists of thirteen tests and an initial page used 
for teaching what “cross out” means. Tests 1, 2, 3, 4, 5, 6, 
7, 9, 11, 12, and 13, in order of presentation in book are 
adaptations of tests 30, 7, 13, 15, 12, 19, 24, 25, 33, 17, and 23 of 
the Binet-Simon Intelligence Scale, 1911 Revision (Town trans- 
lation). These tests in order of presentation are: color naming ; 
pointing out of eyes, nose and mouth; pointing out of objects 
named ; discrimination of length ; discrimination of sex ; count- 
ing four objects ; counting thirteen objects ; aesthetic judgment ; 
completion test ; copying of square; copying of diamond. Two 
additional tests are included in series—number eight, a form 
test, and number ten, a completion test. Both of these are 
based upon well known tests. 
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In each of the cross out tests, and all are cross out tests 
with the exception of the drawing of the square and the 
diamond, three chances are given child to demonstrate ability 
to deal with the problem. For each test three pictures are 
presented on one page, but distinctly separated, in all of which 
the fundamental problem involved in the cross but is identical. 
If the child succeeds in crossing out two correctly it is assumed 
that he understands the problem and full credit for test is 
awarded. Exceptions to this rule are made in the color naming 
test and in the aesthetic judgment test, in both of which a 
perfect performance is required. This variation is necessary 
in the aesthetic judgment test because it is impossible to 
arrange the three pairs of pictures in such a way that two 
ugly ones will not be in line with each other, and a child might 
lake two successes by the simple procedure of marking all 
1e pictures on the right or left of the page. The color test 
also gives much opportunity for chance successes and it was 
therefore thought better to require three correct cross outs. 

Partial credits are used only for drawings of square and 
diamond. The individual differences in the drawings are 
so great that a credit of success or failure seems inadequate 
to meet the situation. The partial credits are based on ex- 
aminers judgment of (1) general shape of drawing, (2) the 
accuracy of the angles. Only one attempt at each drawing 
is allowed. 

The performance is scored by means of a point scale. The 
number of points assigned for each of the thirteen tests 
varies with (1) the difficulty of the test, (2) the mental 
age at which ability to perform the test is usually developed. 
The degree of difficulty was experimentally determined 
by trying out the tests on 419 kindergarten children and 
778 first grade children in several of the schools of Kansas 
City, Missouri. The number of children failing in each test 
was recorded and the tests then arranged in rank order 
determined by the number of children failing in each. The 
mental age was assumed to be that determined by Binet for 
the performance of similar tests in the Intelligence Scale. 
Tests number eight and ten, having no equivalent in the Binet 
series, were scored entirely on a basis of rank order. 

The lowest score assigned to any test is six, this is given 
to tests placed at the four year level by Binet and the other 
tests are scored on the principle of one additional point for 
each additional year of assigned mental level. The rank order 
of difficulty made it necessary to score the aesthetic judgment 
test one point lower, and diamond drawing one point higher 
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than the Binet age ranking would indicate. These are the 
only conflicts between Binet ranking and our attained rank 
order of difficulty in spite of the fact that mass instead of 
individual methods were used. In scoring drawing of square, 
four is given for shape and three for four correct angles. 
In scoring drawing of diamond, five is given for shape and four 
for four correct angles. 

The rank order (based on number of failures in each) of 
the thirteen tests, with scores assigned to each is as follows. 
The tests are numbered according to their order of presenta- 
tion in the book, which was determined partially by their 
supposed difficulty, partially by the desire to have the blank 
pages for drawings at the end of book and test series, and 
partially for the printer’s convenience in arranging the color 
pages. It is probably of some practical advantage in giving 
the tests to have the more difficult ones distributed through 
the series rather than grouped. 


Order of difficulty Test No. Test Score Page 





6 


ms 
NwOO=—MmUbM 


1 Q 
2 3 
3 2 
3 5 
5 12 
6 9 
7 6 
8 7 
9 


_ _ 
o~-o& 
—_— 


— 
— 


oo 
lel 
CSCO OCOMBNININADAG 


—t et et 
COWwD 


PROCEDURE 


As indicated by the name of the test series “ Picture Game,” 
this series is a game and should be conducted in the spirit of 
one. No notion of a test should be allowed to enter the 
children’s minds. The children should be seated at as great 
distances from one another as possible in order to prevent 
copying. This is particularly necessary with kindergarten 
children. Not more than twenty kindergarten children should 
be handled in one group. The tops of desks should be cleared 
and each child provided with a black crayola. 

When all is ready the examiner asks the children whether 
they would like to play a game, and when they assent she 
tells them that each will have a little picture book to play 
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it with. She then distributes the books telling the children 
to place them on their desks so that the rabbit is looking at 
them, and to leave them on their desks during the whole 
game. Allis then ready and the examiner proceeds as follows: 
“Do you see the rabbit? Do you see the rat? Cross out 
the rabbit. When I say ‘cross out’ I mean, put a line through 
it like this.” Demonstrate on blackboard and on sample book 
which you hold up before the class. After rabbit has been 
crossed out say, “ Do you see the duck? Do you see the picture 
of the book? Cross out the picture of the book.” This first 
page of the book is not a test but is used for instruction in 
crossing out. The test series is not begun until the examiner 
is assured by actual inspection of each child’s book that all 
the children know what is meant by “cross out.” This is 
the only bit of knowledge which is required for the manual 
execution of all the cross out tests. When it is determined 
that each child knows what is meant by “cross out” proceed 
as follows, allowing ample time for crossing out between each 
instruction: “ This little book is full of pictures, and on each 
picture I am going to ask you to cross out something. You 
cross out just what I tell you to, and do not cross out any- 
thing else. Now we will begin. Are you all ready? Turn the 
page. : 

Test 1, page 2—“Cross out the red. Page 3—“On the 
next page cross out the blue. Turn the page. 

Test 2, page 4—“ See the pretty little girl at the top of the 
page? Cross out her nose. 
See the little girl just below her? Cross out her eyes. 
See the last little girl? Cross out her mouth. 

Test 3, page 5—“ At the top of the next page you will see 
a picture of a room. Do you see it? Cross out the bed. 
Below that picture you will see another picture. Cross out 
the boat. 
At the bottom of the page you will see another picture. Cross 
out the key. Turn the page. 


Test 4, page 6—‘ There are two pencils at the top of this 
page. Everyone see them? Cross out the longer pencil, the 
longest pencil. 

Below the pencil there are two rulers. Cross out the longer 
ruler, the longest ruler. 

At the bottom of the page there are two knives. Cross out 
the longer knife, the longest knife. 


Test 5, page 7—“ At the top of the next page you see a little 
boy and a little girl. Cross out the boy. 
In the next, picture cross out the man. 
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In the next picture cross out the girl. 
Turn the page. 

Test 6, page 8—“ At the top of the next page you will see 
some chains. Do you see the chains? Cross out the chain 
that has four links. 

Below the chains there are some apples. Do you see the 
apples? Cross out the pile of four apples. 

Below the apples there are some pears. Cross out the pile of 
four pears. 

Test 7, page 9—“ At the top of the next page you will see 
some acorns. Do you see them? Cross out the line of thirteen 
acorns. 

In the next picture you see a lot of little ducks. Cross out the 
line of thirteen ducks. 

Below the ducks there is a picture of blocks. Cross out the 
line of thirteen blocks. 

Turn the page. 

Test 8, page 10—“ At the top of the page you see four 
blocks—a circle, a triangle, a cross, and a square. There are 
heavy lines under them and just below five blocks. Four of 
these are like the blocks at the top but there is one left over, 
which is not like any at the top of the page. Cross out this 
one that is left over, the one not like any block at the top of 
the page. 

Below this picture is another picture of five blocks. Four 
of these are just like the four at the top of the page, and there 
is one left over. 

Cross out the one that is left over. 

At the bottom of the page there is another picture of five 
blocks, four of these are just like the four blocks at the top 
of the page, and there is one left over. Cross it out. 

Test 9, page 11—“ At the top of the next page you will see 
two ladies, a pretty lady and an ugly lady. Cross out the 
ugly lady. 

In the next picture you see two horses. Cross out the ugly 
horse. 

In the next picture you see two men. Cross out the ugly man. 
Turn the page. ; 

Test 10, page 12—“ At the top of the next page there are 
pictures of three objects. Two of them are something alike 
and one is different. It is not like the other two. Cross it out. 
Below this picture there is a picture of three people. Two 
of them are alike. The one is not like the other two. Cross 
it out. 
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At the bottom of the page you see the picture cf three animals. 
Two of them are alike. One is not like the other two. Cross 
it out. 

Test 11, page 13—‘ At the top of the next page there is 
a picture of acow. Do you see it? When they drew that cow 
they did not quite finish her, they left something out. Put 
a mark where it should be. 

Just below the cow there is a picture of a boy’s head. When 
they drew that head they did not quite finish it, they left 
something out. Put a mark where it should be. 

At the bottom of the page there is a picture of a house. When 
they drew that house they did not quite finish it, they left 
something out. Put a mark where it should be. 

Turn the page. 

Test 1, page 14—“ Cross out the green. 

Test 12, page 15—"** You see the next page is an empty page. 
| am going to draw something on the blackboard, and I want 
you to draw it on this empty page, just as nicely as you know 
how.” (Draw a square on blackboard.) 

“ Turn the page. 

Test 13, page 16—‘‘ The next page is empty too. Now | 
am going to draw something else on the board, and I want 
you to draw it on this empty page just as nicely as you know 
how.” (Draw diamond on board.) 

“Turn your books over so that the rabbits will be looking at 
you again. 

Write your name on the line marked name.” 

After name is written, say—‘‘ Now write how old you are.” 

If children have not learned to write, have teacher write 
names and ages. 


EXPERIMENTAL RESULTS 


Twelve kindergarten classes, containing in all 419 children, 
(average age, 5.5) and twenty-two first grades, containing in 
all 778 children (average age 6.9) were tested by the writer. 
The number of individuals failing in each test was counted 
and recorded in terms of percentage of grade group. The 
average total score, with its mean variation and probable 
error, and the median, were computed for each separate class 
and each grade group. Finally the individual total scores of 
children in each grade group were thrown into percentile 
tables in order to show what scores the 0, 10, 20, 30, 40, 50, 
60, 70, 80, 90, and 100 percentiles of each grade group secured. 
All of these results are here presented in tabulated form. 
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A group of second grade classes were also examined, but 
the tests proved to be so far below the level of ability of the 
second grade children that it is not worth while to present 
the results. 

The results here given confirm the impression gained by 
the writer in conducting the tests that the Picture Game is 
adapted to use with kindergarten and first grade children, 
and that by its use one can obtain a fairly good classification 
of children in these two lower grades. 


PERCENTAGE OF CHILDREN FAILING IN Eacu TEstT 


Rank order Kind. Child. First Grade Child. 
of test Test No. failing—per cent failing—per cent 
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The rank of tests 4, 3, 2, and 5, the four lowest in rank and 
of tests. 13,.11, and. 8, the three highest in rank and of test 
9-are identical in kindergarten and first grade groups. The 
differences of rank order are caused by tests 6 and 7, the two 
counting tests, and are probably due to the facts that count- 


CLass AND Group ScorES—KINDERGARTEN 
Class Numb. Child. Ave. Age Ave.Score M.V. P.E. Median 


54.2 22.9 19.35 
62 18.1 15.29 
41.7 13.94 
38.5 17.07 
37.5 16.48 
41.4 12.59 
47.3 14.70 
36.9 17.75 
51.7 18.76 
10 56.9 17.58 
il 56.4 15.72 
12 42.1 14.96 


Totals 566.6 194.19 
47.2 16.18 
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ing is still a novel process for the kindergartners and has 
already become almost automatic with the first grade children. 
Test 12 is displaced in the first grade series by tests six and 
seven which take precedence. The adopted rank order was 
obtained by pooling the results obtained from kindergartens 
and first grades. This order is used in the above table. 


PERCENTILE SCORES—KINDERGARTEN 
Percentiles Scores 


CLass AND Group ScorEs—First GRADE 
Class Numb. Child. Ave. Age Ave.Score M.V. P. E. Median 


81.5 8.03 5 
66.8 14.87 
70.5 12.59 
7.52 
10.90 
12.25 
8.19 
11.58 
9.55 
6.08 
8.96 
12.00 
12.34 
11.66 
11.41 
10.90 
12.42 
9.55 
10.14 
10.14 
13.10 
12.93 
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PERCENTILE SCORES—First GRADE 


Percentiles Scores 


The tests were also used with a number of Italian and a 
number of colored classes. The results obtained from both 
differed so widely from those here reported that the writer 
omitted them from the compilation and reserved them for 
future report. A study of the tables here presented reveals 
in several instances great variation in performance between 
parallel grades in different schools. A study of the causes 
of these differences is of interest from many angles, and 
brings to light many factors of practical importance for these 
Kansas City Schools. The present article is designed simply 
to present the Picture Game as a new and possibly useful form 
of test, to describe it and give explicit directions for its use. 
To preserve the clarity of this purpose discussion of the 
problems mentioned in this paragraph is here omitted. 
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PICTURE GAME 


Name 
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Norte.—On test sheets blocked spaces are colored as indicated. 
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Note.—On test sheets blocked spaces are colored as indicated. 
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Note.—On test sheets blocked spaces are colored as indicated. 


The booklets referred to above may be obtained from Clara Harrison 
Town, 2328 N. 22nd St., Philadelphia, Pa. 





STANDARDIZATION OF THE WHIPPLE-HEALY 
TAPPING TEST 


Prepared under the Direction of Amy Hewes, by Etsie Bonitz, AcGNEs 
Davis, Persis Moore, CHARLOTTE Porter, FLORENCE TURNER, 
Students in the Course in Statistics at Mount Holyoke 
College 


The present study is a standardization of a test for measur- 
ing speed and accuracy of eye-hand co-ordinations. It was 
made in the spring of 1921 by five students in the course in 
Statistics at Mount Holyoke College. The central tendency 
was estimated for each age group in a total of 2,253 children, 
ranging in age from seven to seventeen years, in order that 


TABLE 1 
NUMBER OF CHILDREN TESTED BY AGE GROUPS 


Number 

Age in Years All Records Chicago Holyoke 
and and 

Total Boys Girls Boston South 

Hadley 


Total 2253 1444 809 55 398 





7 and less than 8 155 81 74 21 134 
8 and less than 9 156 84 72 48 
9 and less than 10 166 97 69 103 
10 and less than 11 159 103 56 103 
11 and less than 12 158 119 39 126 
12 and less than 13 180 136 44 177 
13 and less than 14 141 67 206 
14 and less than 15 224 98 322 
15 and less than 16 100 331 
16 and less than 17 c 127 317 
17 and over 63 101 





the performances in any given case might be rated and used 
as one of a group of results on which to base vocational 
advice. 

The data comprised the records for 1,855 tests made: in con- 
nection with the work of the Juvenile Psychopathic Institute 
in Chicago, and the Judge Baker Foundation in Boston, in 
both places under the direction of Dr. William: Healy and Dr. 
Augusta F. Bronner, and the records for 398 tests made of 
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children in schools in Holyoke and South Hadley for the pur- 
poses of supplementing certain of the age groups where the 
numbers were not sufficient. Care was taken to give the latter 
in schools in different sections of the city in order to get as 
varied a selection of children as possible. The records were 
all of children classified as normal mentally. 

It was at first thought necessary to secure a minimum of 
150 cases in each group, but later, when it was desired to make 
a comparison between girls and boys, some experimentation 
led to the belief that 80 were sufficient for a fair basis, for 
the reason that the median for this number remained un- 
changed when the group was increased. 


THE TEST 


The purpose of the tapping test is to determine speed and 
accuracy of eye-hand co-ordinations of the child. For the test 
a sheet of paper is used on which is printed a rectangular form 
divided into 150 half inch squares, as shown in the following 
diagram. 
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The test is given in a room where there is nothing to disturb 
or distract the child, and the examiner tries as nearly as possible 
to put him at his ease. The subject is seated at a table and told 
to tap across the paper (placed as illustrated) once in each 
square, as rapidly as possible without making any mistakes, 
the procedure first being demonstrated. An error consists of 
putting more than one dot in a square, touching the line, or 
skipping a square. Thirty seconds are allowed for each trial. 
After the first trial the paper is turned around and the test 
is repeated. 

The best of these two records is used in grading. 


DIRECTIONS 


The examiner says, “ This is a test to see how well you 
can use your hands. This is how you do it. When I say, 
‘Go.’ you are to put a little tap like this (show how the taps 
are rapidly made across one row and back on the next line), 
one in each square, never missing a square and not touching 
any lines. See in how many squares you can tap till I say 

Sit comfortably with your arm the way you want 


Have a stop-watch in hand and be in position to closely 
observe performance for errors. Start watch exactly as you 


say “Go,” and say, “ Stop,” when watch is exactly at 30”, 
noting where pencil of subject is at this time. 

After first trial turn paper upside down, erasing dots if space 
is needed, and get second record for 30” on the same sheet, 
saying, “ Now, let us see if you can do better a second time.” 

If the subject is too careful about making large dots or 
lines warn him that only light taps are necessary. Endeavor 
to stimulate if lethargy is shown on first trial. If hesitation 
because of error at any time, tell subject quickly to go ahead. 


ScoRING 


In scoring the test three points are taken into consideration: 
the speed, which is expressed numerically as the total number 
of squares tapped; the error, or the number of mistakes which 
are made ; and the index, which is arbitrarily ~~ to be the 
speed minus twice the number of errors. 

In establishing standards for the different _, groups the 
median was chosen to indicate the normal performance of the 
children of a given age. Since one-half of the children stand 
above this point and one-half below, the figure may be taken 
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as descriptive of the ability of the ordinary or normal child, 
and is therefore called a norm. 

The individual performances of the children of the different 
age groups are exhibited in Table Il. The figures show a 
marked and continuous development in motor co-ordination 
as the age increases, with only a slightly Smaller increase in 
the rate of improvement for the older children than for the 
younger. 

TABLE III 
MEDIANS FOR TAPPING TEST 


Medians 
Ages of Children Speed Error Index 
Tested Total Boys Girls Total Boys Girls Total Boys Girls 
7 and less than 8 f d 47 6©§ €¢ 8&8 £6 
8 and less than 9 5% 5 53 1 1 48 46 
9 and less than 10 § 56 60 1 55 : 
10 and less than 11 s ): 70 1 62 
11 and less than 12 ( ) 76 0 67 
12 and less than 13 72 77 1 70 
13 and less than 14 76 : 81 QO 74 
14 and less than 15 82 : 86 79 
15 and less than 16 84 7 90 82 
16 and less than 17 86 f 89 84 2 
0 e9 <¢ 89 
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17 and over 91 9% 90 


It will be observed from Table III that the speed with 


which the task is accomplished is the important factor in deter- 
mining the index to represent the performances of the groups 
of children of various ages, for the number of errors is small 
and appears to have slight relation to age. One error was 
the median for all ages except for the youngest children. If 
the boys and girls are taken separately, it is seen that the 
majority of the girls in three of the groups of older children 
made no errors. As a consequence of the small number of 
errors, the index number follows very closely the general 
trend of the curve for speed. 

A question which is opened by these figures is that of the 
relative ability of boys and girls. The curves show that the 
girls tested have developed motor skill ear?ier than boys, and 
that they maintain a small, but nearly constant degree of 
superiority until the age of 17, approximately, is reached, 
when they are overtaken by the boys. The numbers tested 
are not sufficiently large to warrant generalizations as to the 
relative ability of girls and boys of various ages in other per- 
formances of this kind. They do, however, indicate that the 
field would be a fruitful one for further experimentation, 
especially in view of the importance of the results for prac- 
tical use. 
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Figure B 


MEDIANS FOR THE INDEX BY AGE GROUPS 
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SOME FACTORS OF AESTHETIC JUDGMENT 
By Mrs. ANNE Ross REYMERT 
CONTINUED FROM MARCH ISSUE 
EXPERIMENT XVII 
Series S. Judgment based on degree of suggestiveness of 
an idea,—i.e, the concept of “Spring” in illustrated Spring 
advertisements of similar size, non-colored, from which all 
word matter was removed. The advertisements involved a 
great variety of suggestions, which fact was fruitfully brought 
out by the reasons for choice noted by the observers. 
From the average rankings, which give an idea of the place- 
ment of the pictures: 
Labels 
of pictures Girls Boys 
6 
7 
10 


] 
2 
3 
4 
5 


oo] 


10 


the following individuals correlations with the average group 
rankings of the sex-groups, respectively, are derived: 
GIRLs: 


mf me tk ec ml ad ar gl mr bt ch 


gs 
0.89 0.36 0.64 0.79 0.79 0.27 0.59 0.20 0.24 —0.09 0.67 


0.90 
Boys: 


cc GJ HH LB cs NE LH wD EO DN RG EK 
0.07 0.52 0.50 0.03 0,27 0.85 0.49 0.28 —0.52 0.48 0.38 0.48 


Average correlation of girls as a group 0.52 
Average deviation of girls as a group + 0.27 
Average correlation of boys as a group 0.32 
Average deviation of boys as a group + 0.24 

The correlation between girls and boys as one group=0.50. 





1JIn all the following experiments the observers were requested to 
note the reasons upon which their judgments were based. 
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That the girls as a group agree so much better among them- 
selves, than do the boys among themselves, seems to be due 
among other factors to their having a typical high degree of 
suggestiveness towards “Spring,” (as their reasons for 
choices bring out,—regarding Spring clothing, etc.). Their 
broader range of variation seems (from the reasons given) to 
have its foundation upon a generally wider imagination, as 
regards the above Spring advertisements. Spring is in many 
respects a more decidedly outstanding season of the year for 
girls than for boys, and gives them more suggestions, thus 
making the degree of certainty in choice less. 

Comparing our results in this test with the results of Series 
C., in which suggestiveness of a concrete object was involved 
we see that the sexes there agreed with each other to a much 
greater degree (0.79 as against 0.50 in Series S), and thus 
were On more common ground in their judgments. Also in 
Series C. the girls have a greater variability among them- 
selves than the boys (girls +0.25, boys + 0.21), which seems 
to restate the fact of their having a greater general range 
of suggestiveness, and therefore, a lesser degree of certainty 
in placing the pictures by actual votes. 

From the reasons given for choice it is clear that one and 
the same picture has been given the same rank by different 
observers, often for reasons having very little relation to 
each other. 

Considering the factor of suggestiveness brought into an 
aesthetical judgment then, we may state, that a very complex 


process of judgment evolves which is based upon a variety of 
reasons. 


SoME REASONS FOR CHOICE 
Girls ; 
(i) First appearance of his new straw hat. 
(j) Lambs and young birds most probable in Spring. 

Young chickens just hatched. 

She dressed in Spring clothes, but it is not warm 
enough yet for his straw hat. 

Spring millinery. 

Weddings more frequent in Spring. 

Unsatisfied with winter hat,—he is too warm 

for it. 

The huge raindrops I at once associate with 
April weather. Then, too, the Spring look 
upon the young man’s face helps to carry out 
the idea of April showers. 
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The thing which suggests Spring most in this 
picture is the birds. The gamboling lamb is 
also suggestive of Spring. The hare one 
associates with Easter. 

Chickens are for the most part suggestive of 
Spring. The rabbit and chick one connects 
with Spring. 

The hare trying on a new hat and gazing in the 
mirror remind one of the usual custom of 
buying a new Spring hat. 

The ladies’ skirts blown by the brisk Spring 
winds. 

The young chap is so glad that Spring is here 
that he climbs up into the newly leaved tree. 
The young man is presenting someone with a 
straw hat ; however, I usually associate a straw 

hat with Summer rather than Spring. 

It is an old saying that in Spring young men 
lightly turn to thoughts of love; this sketch 
is, however, more suggestive. 


Spring to me means first showers, then flowers, 
thus (“f” was most suggestive of rain, then 
spring lambs, in “j,” came next, “c” with 
birds, flowers and boys came third, with golf 
came fourth, little children fifth. The others 
mostly suggested Spring styles. 

The airiness. 

The opening of feathers like awakening of Spring, 
with young people taking part. 

Volubility of birds and light-heartedness. 

Bright flowers. 

.£.h.) Excitement and joy of Spring. The first two, 
harmony. 
) Awakening of interest: blossoming out in new hat. 

No reaction. 


EXPERIMENT XVIII 


Series AA. Judgment based on picturesque effect in nine 
scenes illustrating Dutch costumes of similar rich colorings 
in the foreground. As in the foregoing experiments the 
observer’s attention was explicitly directed to one factor, here 
to the picturesque. 
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Average Rankings: 
Labels 
of pictures 


WOONAUSWH 


Individual correlations with their respective sex-group 
rankings : 


mf me tk ec ml ad ar gl mr bt ch gs 
0.28 0.10 0.90 0.02 0.55 0.83 0.81 0.75 0.80 0.71 0.83 0.45 


cc GJ HH LB CS NE LH WD EO DN _ RG EK 
0.42 0.77 O.53 0.63 0.70 0.87 0.43 0.67 0.63 0.33 0.60 0.40 


Average Correlations Average Deviations 


G. 0.59 +25 
B. 0.58 + 13 


Correlation between girls and boys as one group—=0.87. 

Also here there is a very distinctly greater agreement be- 
tween the sexes, than within them, one of the highest through- 
out. 

The high uniformity of judgments, and the comparatively 
small average deviations, as well as the distinctly high correla- 
tion between the sexes as one group, indicate that there has 
been a very similar high uniform aesthetic reaction, pointing 
to a clear-cut impressionistic response. 

From the reasons given for choice it may be inferred that 
the picture is generally reacted to as a whole, as sorne sort of 
an immediate, undefinable oneness, (which fact was also 
corroborated by the oral comments of the observers.) The 
observers were at a loss in searching for reasons after the 
choice was made and reflected longer before doing so. They 
all expressed spontaneous aversion and some positive inability 
to analyze their motives. 

Words like “setting,” “ natural surroundings,” “color effect,” 
“ atmosphere,” etc., all suggest generic homogeneous reaction. 


Some typical reasons for choices: 
Girls: 
I. (a) Presence of windmills—extreme dress of woman 
and child—coloring. 
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Sheep—Naturalness of woman. 

Quaintness of girls—bright coloring—back- 
ground. 

Water scene—dainty coloring—quaint figures. 

Little town in distance—rather elaborate dress 
of woman. 

Coloring—old age and youth. 

Sameness of coloring—children’s activities— 
boats. 

Simplicity. 

Few strokes, atmosphere. 

Atmosphere and motion. 

Atmosphere. 

Motion. 

Judgment was based on color, scenery, character- 
ization. 


1 I believe the women are in a typical attitude 
—that is, of resting just before going to or 
leaving the market, the row boat in the 
background and the sail boat adds pictures- 
queness to the whole. 

The occupations of the people shown here 
in their environment is pleasing. 

The three figures in the foreground give one 
an impression of peace and industry—the 
manner in which they have squatted them- 
selves on the sand is interesting. 

The bright colors in the aprons and skirts 
is pleasing. 

The plain rough clothing worn by the woman 
harmonize with the fishing ships in the 
background. 

The coloring here, although bright, is pleas- 
ing and the gradual dwindling of the line 
marking the water and sky gives me the 
impression of great distance. 

The prosperity of the country is evident. The 
sail boafs as well as the green grass suggest 
picturesqueness. 

There is nothing very striking about this 
picture, except the odd costume. 

I do not like the tranquil, modest expression 
and attitude of the two young women. 








c- 


S 
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II. (a) 1 The most picturesque costume appeals to me 
first—local peculiarities of dress (first 
group. ) 

(b) 2  Peculiarities of surroundings, quaint scenery, 
sheep raising (rural), fishing town—(second 
group. ) 

(c) 3 Striking color presentation, unusual contrast 
of pleasing colors (third group.) 


The reaction reminds one of the definition of William 
James of “ Extensity ” as a primary basic element in sensation ; 
the “ vastness ” or “ bigness” of a primary visual impression, 
as given by him; the extensity of sound volume as introduced 
by Watt. Is there a similar, primary, sensory, aesthetic 
reaction ? 


EXPERIMENT XIX 
Series BB.—Judgment based on Preference, with a view 
toward permanent possession in observer’s room. 
Material_—Colored reproductions of famous paintings. 


AVERAGE RANKINGS 


Labels 
of pictures Girls Boys 
] 2 4 
2 4 9 
3 8 7 
4 9 2 
5 6 6 
6 1 1 
7 7 10 
8 10 8 
9 3 K 
10 5 5 


Individual correlations with the average rankings of the 
sex-groups respectively : 


fm me tk ec ml ad ar gl mr bt ch gs 
0.35 0.44 0.84—0.35 0.60—0.14 0.70 0.87 0.62 0.88 0.76 0.24 
cc GJ HH LB cs NE LH WD EO DN RG EK 
0.50 0.58 0.48 0.50 0.19 0.09 0.45 0.59 0.19 0.82 0.88 0.15 


Average Correlations of the 


Sexes as Separate Groups Average Deviations 
Girls 0.48 + 0.31 
Boys 0.45 + 0.20 


Correlation between girls and boys as one group—0.44. 
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The above results demonstrate that when the aesthetic judg- 
ment, as here, is narrowed down to sharp individual taste, and 
the stimulus pictures have very little in common (except that 
they are all famous paintings of men and women,)—as might 
have been expected,—the individual differences stand out 
distinctly. 

The reasons given range from “the picture that can give 
me ideas, to help me be of better service in the world,” to 
“color, light and shade; technique,” with all varieties in 
between. 


EXPERIMENTS XX—XXI 


Judgment based on most effective portrayal of the feminine 
characteristic, Series CC, and, judgment based on most effec- 
tive portrayal of the masculine characteristic, Series DD. 

Material_—Ten representative portraits of each sex. 

Average rankings by the girls and boys, as separate groups, 
respectively : 

Series CC Series DD 


Labels of Labels of 
Pictures Girls Boys Pictures Girls 


— 
RACUNON WO 


NWWOHK CORDA 
CSODBNAMNAWNHS 


1 
2 
3 
4 
5 
6 
7 
8 
9 
0 


— 
ys 


Individual correlations with the average sunking of the 
Girl group: 


mf me tk ec ml ad ar gl mr bt ch gs 


CC 0.27 0.43 0.65 0.59 050 0.70 0.25 0.71 0.79 0,64 0.73 0.49 
DD 0.31 0.90 0.83 0.22 0.49 0.60 0.79 0.44 0.87 0.55 0.79 0.6) 


Individual correlations with the average ranking of the 
Boy group: 


CC GJ HH LB CS NE LH WD EO DN RG EK 


CC 0.61 0.36 0.18 0.48 0°61 0.98 0.73 0.79—0.01—0.28 0.82 0.58 
DD 0.55 0.95 0.89 0.90 0.68 0.41 0.84 0.82 0.19 0.62 0.77 0,68 


The average correlation of the girls as a group in Series 
CC =0. 56. 

The average correlation of the girls as a group in Series 
DD =0.62. 
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The average correlation of the boys as a group in Series 
CC=0.4%. 
The average correlation of the boys as a group in Series 
DD = 0.69. 
Series CC Series DD 
Average correlations of Average correlations of 
the sex groups with the sex groups with 
their respective group their respective group 
rankings . rankings , Dv. 
Girls 0.56 +0.15 0.62 +0.18 
Boys 0.49 +0.28 0.69 + 0.17 
Correlation between the sexes Correlation between the sexes 
as one group = 0.64 as one group = 0.96 


Again greater agreement between the sexes than within 
them,—in both series——is observed. These two experiments 
bear out a greater agreement in regard to the masculine char- 
acteristic (Series DD) than in regard to the feminine charac- 
teristic (Series CC) within each sex group. And still more 
striking is this agreement when the sexes are taken together as 
one group and compared—this correlation being the highest 
of any experiment throughout all twenty-two experiments. 

In Series CC there is a greater agreement on the part of 
the girl group. In Series DD there is somewhat greater agree- 
ment on the part of the boys. Also the difference in agreement 
on the same sex is approximately equal. 

The boys are much more in agreement on their own sex, than 
the girls are on theirs. 

From the reasons given the choice of the girls were based 
mainly on feminie charm and tenderness ; and on motherliness 
by both boys and girls. Strength, determination and courage 
are the bases of judgment for the masculine characteristic by 
both sexes. 

In regard to the agreement of the individual girl or boy 
with chance neighbors the following tables give a detailed 
account : 
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EXPERIMENT XXII 


Series EE.—Judgment based on degree of impressiveness of 
ten religious pictures. 
Average rankings by the sex groups, respectively: 
Labels 
of pictures Girls Boys 


10 


COON AUSDWNHS 
_ 
BNNOCOUAADO- 


— 


Individual correlations with the average ranking of the 
Girl group: 


mf me tk ec ml ad ar gl mr bt ch gs 
0.27 0.76 O.38 O.54 0.67 0.37 0.28 O.88 0.41 0.62 0.15 0.22 


Individual correlations with the average ranking of the 
Boy group: 


GC CJ HH LB CS NE LH WD EO DN RG EK 
0.62 0.50 0.32—0.36 0.61 0.60 0.42 0.54 0.54 0.78 0.41—0.10 


Average correlations of girls 
and boys as separate groups Average Deviations 


Girls 0.46 + 0.19 
Boys 0.41 +0.23 


Correlation between girls and boys as one group 0.61. 

As might have been expected, there is a great general diver- 
sity in choice here. Looking over the reasons given for the 
main choices by the girls: 

“Christ’s Humility for the Redemption of the World.” 

“The helpless are protected.” ‘ Motherhood.” 

“ Biblical significances and symbolic Characters.” 

“ Death, suffering, sorrow,” etc. 

We see how outspokenly they have been determined by 
biblical religion. 

The reasons by the boys: 

“Impressive surroundings conveying beautiful thoughts.” 

“ Biblical significance.” (The symbolism.) 

“ Death.” 

“ Portrayal of suffering.” 

““ Sympathy and fear,” etc. 
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Biblical symbolism seems to play the greater part in all 
their judgments, and varies with the individual temperaments. 

To get a somewhat rough measure of the degree to which 
the different girls and boys vary around the group standard 
judgment throughout the twenty-two experiments we may take 
the average of each individual’s correlations with his sex- 
group in each experiment and compare this with the average 
of all twenty-two standard judgments of the sex-groups, as 
below : 


AVERAGE CORRELATIONS WITH THE GROUP 
(throughout all twenty-two experiments) 


For THE INDIVIDUAL GIRLS AND Boys 
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Grand average correlation of the individual girls=0 53 


Grand average deviation of the individual girls +0. 26 


From the table we again see the general high correlation 
of each individual subject, both girls and boys, with their 
respective group judgment (the span of the girls being 0.42 
to 0.66,—the span of the boys from 0.31 to 0.65.) The 
individual variability around the group judgment is on the 
whole the same for both sexes although the span for the girls 


is between 0.21 to 0.35, as against that of the boys, 0.16 to 
0.35. 
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The average correlations of the group judgments of the 
sexes as separate groups are both high. However, the standard 
girl group correlation is higher than that of the boy group, 
which all goes to demonstrate that the girls agree better among 
themselves than do the boys throughout the twenty-two experi- 
ments in preference judgments of the kind here presented. 






































Boys: 
cc GJ HH LB cs NE LH | WD} EO} DN} RG/} EK 
0.59} 0.25) 0.90) 0.45) 0.70} 0.88) 0.62/-0.30) 0.90) 0.24/-0.13) 0.40 
0.89) 0.55) 0.52) 0.76) 0.89) 0.13) 0.98] 0.85) 0.44/-0.19] 0.08] 0.21 
0.71} 0.72} 0.72} 0.82) 0.62) 0.41) 0.62) 0.38} 0.41] 0.62/-0.36) 0.64 
0.85} 0.16) 0.79) 6.50} 0.33} 0.37) -0.02] 0.28) 0.50! 0.14) 0.38/-0.32 
0.71) 0.43) 0.49) 0.62} 0.90) -0.07| 0.75) 0.35; 0.42) 0.66/-0.36) 0.48 
0.62! 0.42) 0.76) 0.81} 0.53) 0.81) 0.58) 0.49) 0.87) 0.13} 0.13) 0.47 
0.54! 0.37) 0.90) 0.48) 0.60}; 0.71} 0.84) 0.26} 0.20) 0.82) 0.53) 0.60 
0.89! 0.66) 0.58) 0.38) 0.94) 0.37} 0.01} 0.50) 0.52) 0.72/-0.37} 0.18 
0.82} 0.66} 0.62} 0.10) 0.41) -0.55|) 0.85) 0.72) 0.67/-0.19} 0.19) 0.36 
0.84) 0.36) 0.82) 0.78) 0.28) 0.81) 0.84) 0.61) 0.94) 0.07! 0.37] 0.55 
0.70} 0.81) 0.28) 0.90) 0.89) 0.77) 0.14] 0.78) 0.88] 0.70/-0.41) 0.71 
0.89) 0.53) 0.67) 0.13) 0.78) 0.20] 0.45) 0.21} 0.40) 0.66) 0.58) 0.33 
0.62; 0.54) 0.07) 0.05) 0.54) 0.04) 0.27] 0.47) 0.50) 0.59) 0.60/-0.19 
0.24) 0.22) -0.01; 0.79) -0.03} 0.52) 0.68) 0.47) 0.13} 0.04) 0.73/-0.35 
0.83) 0.59) 0.59) 0.78) 0.88) 0.41) 0.40) 0.82) 0.38] 0.14) 0.75) 0.47 
0.82} -0.07; 0.26) 0.98) 0.87| 0.95) 0.92) 0.89) 0.33) 0.48) 0.09) 0.64 
0.07; 0.52) 0.50) 0.03) 0.27; 0.85) 0.49) 0.28)/-0.52) 0.48] 0.38] 0.48 
0.42} 0.77) 0.53) 0.63) 0.70) 0O.87| 0.43) 0.67) 0.63] 0.33) 0.60) 0.40 
0.50) 0.58) 0.48) 0.50) 0.19) 0.09) 0.45) 0.59) 0.19) 0.82) 0.88) 0.15 
0.61; 0.36) 0.18) 0.48) 0.61) 0.98) 0.73) 0.79/-0.01/-0.28) 0.82) 0.58 
0.55; 0.95) 0.89) 0.90) 0.68) 0.41) 0.84] 0.82) 0.19) 0.62) 0.77) 0.68 
0.62} 0.50} 0.32) -0.36) 0.61) 0.60) 0.42) 0.54) 0.54) 0.78] 0.41/-0.10 
S. 14.33} 10.16) 11.86) 11.51} 13.19] 10.56} 11.21}11.47) 9.51) 8.38) 6.76) 7.37 
C. 0.65) 0.46) 0.54) 0.52) 0.60) 0.48) 0.51) 0.52) 0.52) 0.38) 0.31) 0.34 
A.D.0.16) 0O 26) 0.21 0.27 0.5% 0.32} 0.24) 0.21) 0.21) 0.31) 0.35) 0.25 
| | | 




















Grand average correlation of the individual boys=0.49 
Grand average deviation of the individual boys==+0.25 


Grand average correlation between girls and boys as one group=0 .68—with an 
average deviation of—=+ 4. 


See also Tables, pages 133 and 134 


The deviation from the standard sex-group average correlation 
is practically the same for both sexes as separate groups, but 
only slightly higher than one-half of this when the sexes are 
taken together as one group. The latter plus the standard 
group average correlation between the sexes as one group, 
0.68, shows a very high degree of agreement between the 
girls and boys, the correlation of the mixed-group being 
throughout the twenty-two experiments a conspicuous sub- 
stantiation of the fact of greater agreement between the sexes 
as one group than within them as separate groups. 
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From the above tabular arrangement we can readily gain a 
direct idea of the degree of correlation that was found between 
the various experiments that suggest a grouping with respect 
to relation and similarity in values tested or values that suggest 
types. The span is from 0.59 to 0.74 (neglecting experiments 
S and AA which show the extremes of 0.50 and 0.87.) 

The following tabular arrangement of the correlations of 
the (1) girls, (2) boys, and (3) the sexes as one group, in 
gradation, according to degree of correlation, gives an over- 
sight over the amount of general agreement there was among 
the three groups in their rating of the various aesthetic values 
experimented upon. 

We see that for the three groups there is a span from 0.26 
to 0.73 for the girl group, from 0.29 to 0.69 for the boy 
group, and one from 0.33 to 0.96 for the sexes as one group. 

The first two have practically the same span, the girls a 
slightly larger, and the third group has the largest span. 


GRADED CORRELATIONS 


...0.73 
..0.70 
...0.69 
...0.62 
. .0.62 
..0.60 
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SUMMARY DISCUSSION 


Series DD, involving the judgment of the masculine charter- 
istic, shows the highest correlation, 0.96, between the sexes 
as one group, and is also the highest of all correlations. This 
may be due to greater popular agreement on the element 
suggested in this experiment. 

Series VII. IV and 1, involved a very simple and clear pro- 
portion relation between two curved and two straight lines, 
and were therefore relatively easy to rank. Their correlations 
are 0.94, 0.88 and 0.84, respectively. 

Series AA, involving a judgment of the picturesque effect, 
shows the high correlation, 0.87, between the sexes as one 
group. This may be due to the fact that this series, depicting 
Dutch costumes in a scenic setting, is popular, simple, rich in 
color effects, attractive, direct in its appeal, and so comparable 
that great difficulties in choosing were expressed by the 
subjects. 

Series C. though not admitting of as ready judgment as the 
last named series, correlates very high, 0.79. The swirls in 
charcoal appeal to the imagination in a concrete, simple, direct 
manner. 

Series A and B (Color), the Arbor and Bird Day magazine 
covers, correlates similarly high, 0.78. The covers as their 
name would suggest were of general human appeal, familiar 
as well as attractive in subject, and illustrative of life out-of- 
doors and of the outdoor world. That the element of color 
played an outstanding part is obvious from the correlation of 
this series when judged on the basis of color. 

Series D and Series B, the former involving judgment on 
beautifully colored non-geometric ceramic art designs, and 
the latter involving judgment on charcoal pattern designs, cor- 
related high and equal, 0.77; this in spite of the fact of their 
great difference. The general attractiveness of the designs as 
well as their direct concrete impressionistic appeal and decora- 
tive suggestiveness may account for the results. 

Series III and VI correlate comparatively high, 0.73 and 
0.70, in the proportion tests, perhaps for similar reasons as 
those above. 

Series I, in which the judgment was based on intricacy of 
design, correlates high, 0.70, especially when it is considered 
that a more complex element is involved. Here the designs 
were also beautifully colored ceramic art designs but geometric 
and involving great intricacy of design. 

Series II, and IV (L) of the proportion tests have a medium 
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high correlation, 0.65 and 0.59 respectively, due perhaps to 
being somewhat less direct in type and having a greater chance 
for confusion. 

Series CC, involving judgment of the feminine characteris- 
tic, shows only a moderately high correlation, 0.64. This may 
be due to a greater diversity of individual taste and ideals in 
regard to the feminine than in regard to the masculine 
characteristic, as borne out by the reasons assigned for choices. 

Series EE, the judgment being based on the impressiveness 
of a set of reproductions of religious paintings, correlates 
medium, 0.61, contrary to expectations from the comments of 
the abservers regarding thier difficulty of choice. The biblically 
significant entered largely in this judgment. 

Series A and B (Design) correlates only medium, 0.56, 
when judged on the basis of design, showing that design has 
a less general appeal than color—a racial development element 
as well as one of individual development. 

Series S, which has its judgment based upon the suggestive- 
ness of the idea of Spring in a set of Spring advertisements, 
involved a multiplicity of suggestions—the same picture being 
frequently chosen for altogether different reasons. Hence, as 
may be expected, the correlation is comparatively low, 0.50. 

Series A and B, when a general simple preference judgment 
is passed upon it, correlates very much lower, 0.43, than when 
the judgment is based upon design,—showing clearly the pre- 
sence of conflicting factors. 

Series A, involved a judgment of technique of a set of 
charcoal drawings of two simple pieces of pottery drawn from 
various yet only slightly different angles. The low correlation, 
0.49, was most likely due to the great homogeneity of this 
series. 

Series BB, involving a judgment in regard to choice as a 
permanent possesion in the room of the observer, had generi- 
cally nothing in common, save that they all were colored re- 
productions of famous paintings. The correlation, con- 
sequently is low, 0.44. 

Series V, which required the finest discrimination of the 
proportion tests, the increments of increase in width being of 
the smallest degree of all these tests—(the figure is the same 
as in Series VI),—correlated only 0.33. 

Series C, which is suggestive of a concreate object, and 
Series S, which is suggestive of an idea (Spring) bear an 
interesting relation to each other, C—=0.79, S=0.50, show- 
ing the part the entrance of the intellectual element plays. 

Series D and Series A and B (Design), which involved 
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design, pure and simple, and design in illustration, respectively, 
bear a somewhat similar relation to each other, D=0.77, A 


and B=0.56. 
The span between the sexes as one group is greater than the 
span for either sex group separately : 
Group = 0.33 to 0.96 
Girls = 0.26 to 0.73 
Boys = 0.29 to 0.69 


The variability in correlations, as well as the genera! agree- 
ment on some series, is readily seen in the above tabulation. 








s3 s3 
ytu [uu 


4q 
yu Iw 
20 tw 13 
yu a) oul 
3q 4} 3 
1 qe 4q 


aw 
se 13 x} 
pe pe Je 
13 oul pe eseIAy 2A0qy 
yo 89 yo 
S144) $149 $1449 


ee 
m 
e) 
= 
~ 
iQ 
m& 


HONACGIANOZ FONASITTALNI TWHANAS) SSANSNOILNGIOSNOZ) 


‘uosLIeduwiod jo sesod.ind 
aiqissod 10} ‘mojeq pozesul SI ‘juswbpnl fo souapyuor pue souassyazut posauab ‘ssausnotjuaszsuod JO 2218p VY} UO SJUBUTLIedxa 
0M}-A}UdM} 9} JNoysno1y3 Jaquawedxe ay} Aq poutes uorssadut jouosiad ay} WO $}90efqns ay} JO Zunel ysnos VW 





A SHIFT OF EMPHASIS NEEDED IN PERSONNEL 
RESEARCH 


By Harry D. Kitson, Professor of Psychology, Indiana University 


The term personnel research means to most people, to 
whom it means anything, the administration of tests—intelli- 
gence, “ special ability,” and trade tests—to persons applying 
for positions. As a matter of fact this is a much too narrow 
view. The analysis of the individual (for which tests are 
the generally accepted though perhaps not the ultimate imple- 
ments) is only half, and the latter half of the business. A 
task of equal importance is the analysis of the occupation to 
which the individual is to be adjusted. This has been almost 
entirely neglected, however. The cult of tests for analyzing 
the individual, with its formalized ritual, has so dazzled its 
devotees as to blind them to the possibility of another method 
of approach to personnel problems. The unfortunate conse- 
quences are not merely the neglect of the indispensable analysis 
of occupations but also a stagnation in the development of 
tests. 

For it must be admitted that despite the great attention paid 
to tests, they are contributing but little to industry, in terms 
of either bulk or quality of service. There is much talk 
about them, but it is in terms of what might possibly be 
accomplished if they were developed along the line of a par- 
ticular theory of vocational guidance. The few cases in 
which they have been practically used are conspicuous because 
of their rarity; and have not generally been followed up as 
a regular matter of industrial routine. One who views the 
matter dispassionately, then, must admit that in spite of the 
attention tests have received (in talk) they have not made 
any appreciable progress in industry during the past five years. 
Despite the rose-colored dreams that are painted of an indus- 
trial system wherein each individual shall be placed on the 
road to success by means of a dramatic series of psychological 
tests, the application of tests to occupational situations con- 
sists in following a stereotyped prescription that brings no 
cure and no change. In the psychologist’s own jargon it may 


be said that psychological tests in industry are on a distinct 
plateau. 
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In order to escape from this plateau the industrial psy- 
chologist may well follow the prescription given by his col- 
leagues in the psychology of learning, namely, change his 
method. Instead of seeking to solve the problems of per- 
sonnel work by testing the individual with artificial, extraneous 
tests, let him approach the operations of the occupation as the 
worker actually has to perform them. By doing so he will 
not only fulfill the demands of an adequate program of per- 
sonnel research, but he will also probably unearth some method 
that will show the way off the plateau upon which tests now 
languish. 


This idea of applying psychological technique to the analysis 
of occupations is not new in theory. The public interested in 
vocational guidance have urged it under the name, vocational 
analysis, meaning the analysis of the professions and trades; 
or under the name, occupational analysis, meaning the analysis 
of specialties within the trades. Within recent times employ- 
ment managers have been urging it under the name Job 
Analysis, meaning the analysis of a particular operation on 
a particular piece of material or a particular machine. Almost 
all who have faced the problems of personnel administration 
have recognized the necessity of this analysis, and particularly 
its psychological nature, but very few have actually attempted 
it, and they with only scattered efforts. As usually conceived, 
the psychological analysis of an occupation' consists in de- 
scribing the occupation by means of rough observation on the 
part of a psychologist in consultation with employment man- 
ager, foreman, expert workman and others. The vsychologist 
may even perform the operation more or less haltingly himself 
in order to secure “local color.” After some such rule of 
thumb observation the occupation is split up into a series of 
“unit operations.” These are then embodied, for purposes 
of selecting fit workers, into a statement called the Job 
Specification, for which outlines of various kinds exist.2 An 
analysis according to such a pattern may be of value for some 
occupations and some circumstances, but it is surely inadequate 





1 It is of course understood that a complete analysis of an occupation 
requires description in terms of the physical, physiological, social and 
economic phases as well as‘of the psychological. 

2 See, for example, H. C. Link, Employment Psychology, New York. 
Macmillan, 1919, p. 261. 

Ordway Tead and Henry C. Metcalfe, Personnel Administration, 
New York, McGraw & Hill, 1920, p. 264-5. 

Franklyn Meine, Job Specifications, Bull. No. 45, Fed. Bd. for Voc. 
Education, Washington, D. C., Nov. 19, 1919. 
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as a sole instrument of scientific personnel research in the 
broad sense. It should be amended in accordance with three 
principles of scientific method: 


1. Analyze the occupation in terms of itself. 

As at present conducted, most attempts at occupational 
analysis violate this in one of three ways. 

a. According to one method greatly favored, the psychologist 
gazes at the occupation for a short time and then concludes: 
“This occupation requires concentration of attention, verbal 
memory, reasoning about new situations, etc. Here are tests 
a, b, and c, which measure these several activities (sic!). ] 
will administer these tests to workers in the occupation and 
choose those tests wherein scores show the highest correlation 
with standings in occupational success.” The “ teste1” activi- 
ties constitute the activities of the occupation, in the eyes of 
the analyst; and the procedure is regarded as a piece of 
occupational analysis. Upon second thought the fallacy of 
this procedure is grossly apparent. It really delineates the 
occupation only in terms of extraneous tests, not in terms 
of the work itself. And it can be justified only on the 
obsessing hypothesis that tests with their wabbly indices of 
correlation are the most reliable tools of analysis. 


b. Another practise much in vogue is to make a list of psy- 
chical traits supposedly required in an occupation—industri- 
ousness, affability, neatness, patience, assertiveness—and call 
them the analyzed components of the occupation. The chief 
objection to the listing of such traits is that they are abstract, 
characterizing a number of occupations and yet no occupa- 
tion in particular. Again they are general; every occupation 
requires them in some degree. Finally, it is not yet estab- 
lished that a trait like neatness, for example, will express 
itself equally in all lines of activity. The greater likelihood 
is that each occupation involving it will require it in a specific 
form. 


A good example of an attempt to overcome this last objec- 
tion is furnished by an insurance company that desired to 
analyze out the psychical characteristics brought into play 
in the best quality of insurance salesmanship. Having dis- 
covered by trial the futility of using such general terms as 
those just mentioned, the company has attempted to formulate 
the required traits in terms of what the best salesman does 
in selling insurance. Thus, instead of scheduling a mythical, 
abstract, general trait like “ dominatingness,” the analyst asks 
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for a rating in this manner: “ Does he (the best salesman) 
dominate the interview most of the time ; half of the 
time - almost none of the time This method 
is not perfect; it involves a danger to be mentioned presently ; 
it is only tentative; but it has the virtue of avoiding general 
terms that mean nothing; and it shows a praiseworthy attempt 
to analyze the occupation in terms of itself. 

c. A third questionable practise among the current attempts 
at occupational analysis is that of seeking the components of 
an occupation in the opinions of expert workers. One of the 
most thoroughgoing applications of this technique is that of 
Martha Ulrich,* who has devised a schema consisting of a 
number of traits, which she presents to professional men— 
doctors, lawyers, etc.—with the request that they rate each 
trait according to the degree to which they think it essential 
in their occupation. Thus: 


1 desirable —1 undesirable 

2 very important —2 disadvantageous 

3 absolutely indispensable —3 very disadvantageous 
O unimportant 


More than 100 traits are scheduled, of the sort condemned 
in (b) above, such as: visual, auditory or motor type of 
imagery; ability to make new associations easily; ability to 
think in abstract terms; a strongly developed emotional life, 
etc. The judgments of all are then summed up to give a 
numerical weight to each trait as a component of each vocation. 

This technique in its various guises constitutes a most pre- 
carious tool of vocational analysis. The dangers of subjec- 
tive data that lurk in it emphasize anew the need for the kind 
of analysis urged in this paper, namely the analysis of an 
occupation in terms of itself instead of in terms of psycho- 
logical tests, abstract non-existent general “ traits,” or opinions. 


2. The second principle is, Express the requirements of 
the occupation in quantitative terms. For example, in describ- 
ing the occupation of proofreading, one should, among other 
things, measure the output of a standard group of workers, 
and upon this basis, state the median requirement to be a 
speed of 900 lines per hour (of a certain kind of material) 
with no more than one error overlooked every 35 lines. Or, 
to change the illustration, one might state the speed required 





3 Die psychologische ‘Analyse der hochen Berufe als Grundlage einer 


kuenftigen Berufsberatung. Zeitschrift fuer Angewandte Psychologie, 
13, 1918, pp. 1-36. 
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in linotyping at the upper ten percentile, to be 8,300 lines 
per hour with no more than one error in every 38 lines. Or 
one might describe typewriting of grade A as an output of 
100 letters per day, B, 80 and C, 60.4 

3. A third requirement is that the analysis should be made 
minutely. Perhaps the rough, non-quantitative kind now prac- 
tised is adequate as a basis for selecting workers for certain 
gross unskilled occupations. But when one takes the pains to 
measure the reaction time of an applicant in fractions of a 
minute as one does in tests, one should have measured the 
object to which the worker must react (the occupation) with 
equal minuteness. 

There are still other considerations that demand that occu- 
pational analysis should be made minutely. Analysis is neces- 
sary for purposes other than merely selection of workers. 
It may be used for discovering wasteful and fatiguing opera- 
tions, as when Taylor found that to analyze such a gross job 
as shoveling required measures involving ounces; and Gil- 
breth found that to analyze the folding of handkerchiefs re- 
quired chronocyclegraphs showing movements lasting but a 
fraction of a second. Again, analysis may be made for the 
purpose of discovering what factors are involved in learning 
an occupation, as in the analysis of telegraphing by Bryan and 
Harter and that of typewriting by Book. A more recent 
attempt by the writer in the field of proofreading illustrates 
the same requirement. The particular aim of the investiga- 
tion was to secure a standard of eye-movement. The estab- 
lished technique for the measurement of eye-movements was 
employed, which permitted their reproduction in terms of the 
number and length of pauses. Table I shows these averages 
in fiftieths of a second for good and poor proofreaders reading 
the same twelve lines. 


TABLE I 
Average number of Average length 
pauses per line M. V. of pauses M. V. 
Good 7.4 .48 17.1 1.66 
Poor 11.0 .00 19.3 3.64 


“From these results we may draw the following con- 
clusions: (1) a good reader makes on the average fewer pauses 
than a poor one—7 vs. 11. (2) A good reader makes on the 

4 Here again, the reader is reminded that the same kind of quanti- 


tative statements should be made on behalf of the non-psychological 
characteristics of an occupation. 
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average shorter pauses than a poor one—17 vs. 19. (3) A 
still more important matter is the demonstration of differences 
in regularity. The good proofreader reads very regularly. 
His eye-movements are so regular as to be almost rhythmic, 
with respect to both length of line and number of pauses. 
The poor proofreader, however, reads irregularly. These dif- 
ferences may be quantitatively stated in terms of the mean 
variations. The small mean variations in the record of the good 
reader indicate that he paused practically the same number 
of times in each line—on the average no more than eight 
(7.4+-.48) times, and no less than seven (7.4—.48) times. 
The poor reader, however, paused in one line as many as 
thirteen (11.0+2.0) times and in another line only nine 
(11.0—2.0) times. In length, the pauses varied between 18.8 
and 15.4 in the case of the former, and between 22.9 and 13.7 
in the case of the latter.” 


RESUME 

Such are the three principles that should be followed out in 
any adequate program of occupational analysis; and such 
are the methods that should be employed. This rigid régime 
is admittedly arduous. It requires that the psychologist who 
would engage in personnel research must leave the shelter of 
his academic laboratory and enter the arena of occupations. 
He need not necessarily don a mechanic’s cap or a mason’s 
apron, but he must seek his data in the activities of the worker 
at work. He need not necessarily invent a battery of new 
instruments for the measurement of these work-activities. 
He may secure many of the necessary measures from the 
records of well-organized industrial and commercial estab- 
lishments. There are vast quantities of such measures avail- 
able to one who will gain the confidence of live business 
executives and show them by results that personnel psychol- 
ogizing as described above will benefit not only personnel work 
in general but also their establishment in particular. 

It may be objected that this program of minutely measur- 
ing occupations in terms of themselves is going to take a long 
time, and that the work of vocational guidance cannot wait, 
hence must proceed with the dubious makeshifts (particu- 
larly tests) described above. The answer is that to search 
for truth always takes time but is more economical in the 
long run. Or still more tartly, that tests are not leading to 


5H. D. Kitson, Scientific Method in Job Analysis. Jour. of Pol. 
Econ. 29, June, 1921, p. 512. 
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speedy victory, and judging from the length of the plateau 
they are now on, cannot progress much farther in their present 
form. Hence, the most economical thing to do is to lessen 
the emphasis upon tests, at least temporarily, and place more 
emphasis upon the scientific analysis of the occupation until 
it has reached a stage of refinement equal,to that of the tests. 
Then, perhaps, the way will be clear to improve the tests. 

Again the objection may be raised that even after an occu- 
pation has been pretty well analyzed in terms of itself, the 
way will not have been opened for testing the ability of a 
novice aspiring to the occupation, for of course one could not 
be expected to perform the skilled operations required in the 
occupation before learning them. It is freely acknowledged 
that the reforms here advocated do not seem to offer a ready 
solution to the problem of selecting a candidate for an occu- 
pation by tests. Still it is possible that after a truthful analysis 
of the occupation is made, the way may be open for the 
development of the needed technique of vocational selection 
and guidance. One who would be bold enough to defy con- 
ventions might suggest further that perhaps it will turn 
out that tests are not the best tools to use after all. It is 
conceivable that a better way may evolve out of a well-directed 
search for light. 

All apart from its bearing upon tests, however, the scien- 
tific analysis of occupations here advocated is bound to result 
in a number of benefits to personnel work conceived in its 
broader sense: 1. It will reveal errors and wastes that may 
be present in an operation, thus reducing the worker’s fatigue 
and bringing profit to both worker and employer. 2. It will 
furnish basis for establishing standards of performance that 
may be demanded of employees. 3. It will assist in scien- 
tifically setting a wage consistent with the amount of work 
done. 4. In connection with jobs involving the use of machin- 
ery it may give information regarding more effective ways 
of adjusting, operating and maintaining machines. 5. The 
cost department may use accurate measures of the units of 
an occupation in estimating more closely the cost of new jobs 
about to be contracted for. 6. Analysis may conceivably be 
used in arranging harmonious grades and routes of promotion, 
showing the points of overlapping in the duties and quali- 
fications of various jobs, and thus indicating at what stages 
workers traveling various routes may be interchanged. 7. 
Scientific analysis is an indispensable preliminary to the inven- 
tion of a trade test. 8. Finally, it is a prerequisite to adequate 
occupational training. 
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In short, a scientifically defensible theory and administra- 
tion of occupational analysis will result in many benefits not 
specifically foreseen; as happened in England when the Indus- 
trial Fatigue Research Board, organized during the war in 
order to “ promote better knowledge of the relations of hours 
of labor and of other conditions of employment including 
methods of work to functions of the human body,” discovered 
facts and developed techniques that have since been found 
directly applicable to the solution of problems other than those 
at first proposed. 

The secret is that under the plan here advocated, the per- 
sonnel psychologist will be working not with spurious extrane- 
ous materials, but with the actual stuff of occupations. And 
when he sheds obsessing theories and seeks for facts in the 
realm of reality he is bound to discover truth even if not in 
the form expected. 





THE USE OF THE PERSONAL HISTORY BLANK AS 
A SALESMANSHIP TEST 


By Dorotny B. GotpsmitH, The Guardian Life Insurance Company 
of America 


This study was made to determine whether the items of 
a personal history blank could be used to predict the success 
or failure of a salesman. That is, whether a weighted, quan- 
titative score could be given to the answers to questions upon 
a history blank, in order that it might be used as an elimina- 
tion test. To accomplish this, the scoring of the blank had 
to perform two functions,—(1) it must eliminate failures, 
and (2) it must not eliminate successes. The chief purpose 
was, therefore, the establishment of a critical score, a score 
below which would fall the failures and above which would 
lie the successes. 

The attempt embodied in this study was successful; a lower 
critical score was established at a point which for the 502 
blanks studied, eliminated 54% of the failures, but left 84% 
of the successes with the company. For this group the estab- 
lishment of an upper critical score was unnecessary. 

In this experiment the personal history blanks of 502 sales- 
men of The Guardian Life Insurance Company of America 
were studied. For the life insurance field in particular, though 
it would hold good in some measure in the case of any large 
organization, there is a peculiar advantage in adapting the 
personal history blank to this purpose. In this business, 
branch offices are maintained all over the country, and it is 
an extremely difficult task to persuade the branch managers 
to use tests, and a well nigh impossible one to have the tests 
given under standardized conditions. The personal history 
blank, however, has for some time been a routine part of the 
application for a sales license. As its name implies, it con- 
tains information concerning the applicant’s personal history 
and past experience. Almost every firm uses a blank of some 
description and its completion presents no novel task. The 
manager has become accustomed to having this blank com- 
pleted by his new men, and, in turn, to sending it on to the 
Home Office. In the past, this blank had been required for 
every agent of the company under discussion. There had 
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been no attempt, however, to give it any scientific treatment ; 
the personal history blank had served rather, in the main, 
merely to strengthen or weaken the general impression made 
by the applicant. Certain items whose presence or absence 
were deemed significant were checked, and the remainder of 
the blank was taken as a whole to determine by its general 
tone whether an applicant should be licensed or dropped as a 
possibility. 

To secure the proper scoring for the blank, the results of a , 
study made about a year ago on 34 salesmen were taken as a 
tentative guide." Fifty history blanks representing three 
classes of agents whose production records were known were 
selected. These classes were (1) those who were failures, 
(2) those who were borderline cases up to moderately suc- 
cessful, and (3) those who were successful. The criterion 
of success was the amount of insurance paid for during the 
first year after the man was licensed. (We had already 
found out that the first year’s production was a very good 
index of future production.) It would be well here to men- 
tion that the only factor which determined the selection of 
the 502 blanks was whether or not the agent’s first year’s 
production record was available. The blanks for these three 
classes of men were then studied and the essential or signifi- 
cant items which varied with the success of the agent were 
selected. Each blank was then graded according to the tenta- 
tive scoring, and the total thus obtained was checked against 
the man’s production record. In accordance with these results, 
the scoring was revised, and the same process repeated with 
25 blanks selected at random. We found that those whose 
scores on the blank’ were low were also low in production, 
and that with increasing production the scores tended to be 
higher. The correlation, of course, was not a perfect one, 
but we were interested rather in establishing the lower critical 
score than in correlations. Inasmuch as the personal history 
blanks selected at random were well distributed into our three 
classes by the scoring which we had determined upon, we 
selected this scoring as our final method. The weighted values 
thus assigned to the significant items were as follows: 

Age . B29 +2 
18-20 —2 30-40 +3 
21-22 —) 41-50 +1 
23-24 0 51-60 0 
25-27 +1 Over 60 —1 





1 Report A—1921, Scoring of Personal History Blanks. Bureau of 
Personnel Research, Carnegie Institute of Technology. 
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Marital Status Occupation 
Married +1 Social I 
Single —1 Unsocial —l 

Service 
Full Time +2 Insurance 
Part Time —2 Carried +1 


, r Not carried —1 ° 
Education 


8 years +1 


10 years +2 Clubs 

12 years +3 3elongs to Clubs +1 

16 years +2 Does not Belong —1l 
Experience 


Previous Life Insurance Experience +1 


Confidence 
Replies to Question: “ What Amount of Insurance Are You Confident 
of Placing Each Month” +41 
Does Not Reply —! 


All of these significant items are the results of study of the 
personal history blanks. They do not represent guesses as 
to what ought to distinguish successful from unsuccessful 
agents, but items upon which successful and unsuccessful 
men actually have differed. Of course, in no single case will 
all these items be significant, various factors will operate in 
some instances to cause certain items to lose their predictive 
value for certain men. It is not upon any single item, however, 
that we will base our conclusions, but rather upon the total 
result from all the items. 

The significance of the items may be explained as follows: 

(1) Education. Through the high school period, school- 
ing seems to be increasingly important to the life insurance 
salesman. When, however, we reach college, other factors 
seem to enter in to render the college man dissatisfied with 
this profession, and for this company, at least, the college 
man is not as successful as the high school graduate. | It 
would be well here to mention the fact that the college group 
is a small one for this company; with a larger one our results 
might well be different. Even with the small group involved, 
there are several notable exceptions. If the college man 
goes into the field with the determination to succeed in life 
insurance salesmanship, and to make it his profession, there 
is no reason why, with his superior training, he should not 
go beyond the high school man. In the past, the difficulty 
has probably been chiefly one of attitude, the college man con- 
sidered the field an experiment in which he might make some 
money while he was casting about for something to do, and 
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did not go in with the real desire to work, and to continue 
working. Hence the results in this study which is, of course, 
based on achievement in the past. 


(2) Age. Up to age 40 at entry, there is a steadily increas- 
ing chance for the applicant to become a successful salesman, 
30-40 being the most favorable period, inasmuch as by that 
time a man has amassed a large enough fund of general 
experience to allow him to see a situation through the eyes 
of a prospect and thus render the prospective buyer the best 
service. After age 40, he generally is too fixed in his ideas 
and habits to be able to make the necessary adjustments upon 
embarking upon a new line of‘ work. 

(3) Occupation. Whether an applicant has been previously 
engaged in an occupation in which he came in contact with 
people is significant as indicative of his probable success as a 
life insurance salesman. The fact of having worked with 
people, of having had to establish connections with them, 
renders the approach to a prospect relatively easy to the man 
who has been engaged in a social occupation, while to his 
fellow applicant who has been busied with individualistic 
work, it is an extremely difficult task. Examples of social 
occupations would be salesman, lawyer, social service worker, 
ex-army or navy man, etc., while of unsocial ones would be 
clerk, mechanic, chauffeur, etc. 

(4) Marital Status. The married man with his increased 
responsibility rendering ‘making good’ more necessary is a 
better prospective agent than the single man. The man with 
dependents may be considered equivalent to the married man. 

(5) Insurance. Whether the applicant carries insurance 
or not is significant as a factor in determining whether he 
will become a good agent, inasmuch as it indicates the fact 
that he is sold on the value of his own product. If he has not 
convinced himself of the value of life insurance, he is not 
likely to convince others. 

(6) Service. The applicant who states that he intends to 
devote his full time to selling insurance is far more likely to 
do his utmost to make a success and to give the profession 
a fair trial. On the other hand, the man who makes life 
insurance salesmanship merely an occupation for his spare 
time whereby he can earn a little extra money, is not likely 
to become a good agent. He will be interested possibly in 
selling insurance, but not in seeing to it that the policy remains 
sold, and he is less likely, through his work, to be in himself 
an advertisement for his company. 
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(7) Clubs. The man who belongs to clubs and enjoys meet- 
ing others ought to be scored positively. He is more likely 
to be able to meet his prospects on their own ground and to 
sell them. This factor is closely connected with the sociability 
of the previous occupation. 

(8) Confidence. The men who reply to the question “ What 
amount of insurance are you confident of placing each month ” 
tend to be better prospects for successful agents. Just what 
guess they hazard is not of importance. But the fact that 
they have confidence enough in their own ability to make 
some prophecy is important. 

(9) Life Insurance Experience. Previous life insurance 
experience is of value to an applicant, although, of course, the 
reasoris for his change from one company to another must be 
ascertained. 

After the total score for each of the 502 personal history 
blanks had been made up, the first year’s production record 
for each agent was secured. The blanks were then divided 
into three groups, (1) those who made a score below 4, (2) 
those who scored between 4 and 8, and (3) those who scored 
above 8. The iowest score actually made was —5 and the 
highest, 14. Each class was then divided into three parts, 
those whose production was below 50 units, those with a pro- 
duction between 50 and 100 units, and finally, those who were 
over 100 units. The number falling into each class was com- 
puted. The following chart indiéates the results of this 
grouping. On the ordinate have been plotted the production 
units, and on the abscissa, the scores on the personal history 
blanks. The double line marks the critical score placed at 4. 

There are 243 men who fall below 4 on the personal history 
blank, that is, 243 men who lie below the critical score indi- 
cated by the double line. Of the 243, 232 also fall below the 
minimum satisfactory production which has been placed at 
50 units. Thus, of men scoring below 4, 96% are failures 
in production. 

The cases to the right of the second vertical are agents who 
scored 8 or over on their blanks. Of these 68 men, 36 had a 
production of 50-100 units or over 100 units, and so, may be 
termed moderately to very successful. Therefore, of men 
scoring above 8 on the history blank, 53% are successful, as 
opposed to 4% of successes among those scoring below 4. 

In between the first and second verticals are the men who 
made between 4 and 8 on their blanks. The results here are 
not as striking as in the other groups. In this division, 




























154 GOLDSMITH 
although 83% are failures, 17% are successes. Should other 
tests be given, they would probably serve to differentiate fur- 
ther among the agents forming this group. 

Looking at the chart the other way, we find that of the 
433 agents whose production was less than 50 units, or in 
other words, of those below the first or lower horizontal, 54% 
fall into the group whose score on the personal history blank 
is below 4. That is, this group of agents scoring below 4, 
the critical score, contains more failures in production than 
do the other two groups of those above 4 put together. If, 
on the other hand, we turn to the group which we may term 
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moderately to very successful, we find that of the 68 agents 
composing this group whose production is over 50 units, only 
11, or 16%, will be found among those in the group scoring 
below 4 on the personal history blank. 

It is evident then that the critical score should be set at 4. 
By cutting out those falling below that figure, we eliminate 
54% of the subsequent failures, while 84% of the subsequent 
successes remain with the company. If the critical score were 
set higher up at 5 or 6, it is true that more failures would 
be eliminated, but at the same time more successes would be 
cut off,—an unprofitable operation. 
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The study of these 502 blanks has, therefore, indicated 
that for a life insurance company, the score on the personal 
history blank bears a positive relationship to the applicant’s 
future success, and that on this blank a lower critical score 
may be set, below which it would not be worth while to license 
an applicant. The same would probably hold true for other 
sales forces in other occupations, but insofar as each industry 
presents a different problem, a different method of scoring 
would have to be worked out in each case. 








THE SELECTION OF A SUCCESSFUL SECRETARY 


A. T. PoFFENBERGER, Columbia University 


One of the high grade secretarial schools for women in 
New York City wished to try out a psychological examination 
as a means of selecting more competent and promising ma- 
terial and for raising the general standard of the school. 
The Army Alpha intelligence test was suggested and tried 
for one year. The data accumulated may be of service to the 
psychologist both by adding to what is already known about 
this examination, and also by indicating some of the diffi- 
culties that are met in applying it in such a practical situation. 

The school is small enough so that each student is well 
known to the instructors and on that account the conditions 
are favorable for making estimates of fitness. Sixty-two stu- 
dents in the school were given the Alpha test. The school 
authorities supplied the following data: 

1. A classification of the students into four groups as to 
their general school standing: Excellent, Very Good, Fair 
and Below the Standard of the School. 

2. A classification of the students into ten groups according 
to their proficiency in each of their four most important 
studies, namely: Typewriting, Stenography, Accounting and 
English. 

3. An order of merit arrangement according to proficiency 
in each of these studies. 

4. A special report on certain students who were consid- 
ered unsatisfactory. 

Table 1 shows the coefficients of correlation (Spearman) 
between the order of the students in each of the four studies 
and in each of the eight parts of the Alpha test; also the 
coefficients of correlation between their order in each of the 
four subjects and that obtained from their total score in the 
Alpha test. If the ranks given for the four studies are aver- 
aged into what might be called a school rank, its correlation 
with the Alpha scores is represented by the coefficient +-.65. 
Most of the correlations seem fairly large. The highest is 
between the test and English (-+-.70) and the lowest is between 
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the test and typing (+.46). The three coefficients in the body 
of the table that are lowest are those between typing and 
“Information” (-++.11) and between accounting and “ Oral 
Directions ” (+.17) and “ Practical Judgment” (+-.10). The 
final coefficient of +.65 is about as high as has been obtained 
between college work and an intelligence test. One might 
then conclude that intelligence plays about as much part in 
secretarial school work as in general college work. 
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Judging from the size of the coefficients of correlation given 
in this table, certain of the separate parts of the Alpha exam- 
ination are as effective for discovering typing and stenographic 
ability as are the special tests which have been tried out for 
that purpose. For example, the following coefficients were 
obtained by Rogers (unpublished) with two of his groups of 
skilled operators. 


TABLE 2 
Typing Stenography 
Rp ee a . 36 
Number checking...... .. hee : .39 .07 
er ee ee . 38 . 34 
Agent-Action...... Pee iibduissiaheh tee . 28 .23 
Action-Agent...... eid se a dleicabes . 26 .47 
Substitution. ..... | A oa) ea! . 20 .40 
I hn. ea dhs ane oares Ss .19 .46 
ERE POOMBEIOOS. 2.5.5 wc cee cene .04 | 
eg 62S son wiisis enn .06 .45 


Without making any detailed comparison, it will be observed 
that most of the Alpha coefficients are higher than those in 
this group. It would seem to be advisable, when using the 
Alpha examination for testing special groups, to work with 
scores of the component tests as well as with the total score. 
Clues as to the measures of special capacity may thereby be 
obtained. 
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Table 3 gives some indication of the degree to which each 
of the eight parts of the Alpha examination is effective in 
picking the good from the poor in each of the four types of 
work, typing, stenography, accounting and English. The best 
ten and the poorest ten students in each of these subjects were 
selected, and their average scores in each of the eight parts 
of the Alpha examination were calculated. The average scores 
were reduced to percents of the maximum scores obtainable 
in the different parts, and these are given in the table. The 
differences between the records of the best and poorest ten are 
also given in the table. 


TABLE 3 


Per cent of Maximum Score 
Part of Alpha > &@ 2 € & 


Best 10 59 81 86 
Typing Poorest 10 34 66 72 
Difference 25 15 14 


Best 10 79 90 
Stenography Poorest 10 70 71 
Difference 9 19 


Best 10 71 88 93 
Accounting Poorest 10 70 74 76 45 
Difference : we at Oe 


Best 10 77 92 96 68 
English Poorest 10 64 68 71 43 
Difference 13 24 25 25 


The school authorities in this case were interested in estab- 
lishing a minimum score for entrance into their school. They 
were further interested in individuals rather than in statistical 
results. They wanted to know how their present students 
whose qualities were known would have been affected by 
establishing a minimum Alpha score at a certain point. To 
answer this question the students were distributed according 
to their appearance in the ten groupings provided by the 
instructors and according’to their Alpha rating expressed in 
steps of ten points each. Such a distribution chart was pre- 
pared for each of the four studies and for a combination of 
these. The results were not satisfying. If the minimum 
score was set at such a level as to retain the good students 
(Alpha score 90) it did not eliminate the unsatisfactory stu- 
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dents; and if set high enough to eliminate the poor students, 
too many of the good students were eliminated. These tables 
will not be reproduced, as a condensed one will serve as an 
illustration. 

Table 4 shows the distribution of students into four classes 
according to their general fitness as estimated by their instruc- 
tors and into eight classes according to their score in Alpha. 
It will be seen that no one who made a score less than 90 made 
a satisfactory record in the school; also that no one who 

















TABLE 4 

Alpha Score 
School 50- 70-| 90- 110- 130 150-/170- 190- 
Standing |69 891109 129 149 169 189 212 
Excellent 4 13/7 1 
Very Good 1 1 8 Si 2 
Fair 1 7 3 3 | 
Below School 1 2 1 2 
Standing | | _— | 





made a score above 171 made a record in the school worse 
than “ Very Good.” If the critical score were set at 90, 
only one of the six unsatisfactory students would be elim- 
inated. If it were set high enough to include half of the 
unsatisfactory students, then two who were satisfactory would 
be eliminated. To eliminate all of the six unsatisfactory stu- 
dents would mean the elimination of twenty-five satisfactory 
students. 

An examination of the cases where the students stood high 
in school work and low in the test shows clearly the impor- 
tance of other than intelligence traits. Most important of 
these is a genuine interest and liking for the work and next 
to this stand loyalty, personal appearance, manner, steadiness 
and other social traits, which the test does not measure. One 
of the two cases of relatively high intelligence and low school 
standing, was a woman who, previously well-to-do, was forced 
by circumstances to earn her livi *. Lack of interest in secre- 
tarial work was obviously the difnculty here. 

The writer believes that performance in an intelligence 
examination such as Alpha may not be an entirely adequate 
and. economical basis for selection of secretaries any more 
than for mill workers and salesclerks. To choose out of all 
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available candidates those of the highest intelligence does 
not guarantee the selection of those having the character traits 
needed for successful secretarial work. Consideration should 
be given to both these aspects of the personality. 

From the psychologist’s point of view it might not be dif- 
ficult to decide upon a critical intelligence score for the pur- 
poses of this school. For instance, a critical score of 130 
would eliminate only 2 above average and 12 below average. 
But here as in most institutions where tests are being estab- 
lished the keenest interest centers upon the individual case 
If one or a few good students be lost, the test may thereby 
be condemned. “If a man have a hundred sheep and one 
of them be gone astray, doth he not leave the ninety and nine, 
and goeth into the mountains, and seeketh that which is gone 
astray? And if so be that he find it, . . . he rejoiceth 
more of that sheep than of the ninety and nine which went 
not astray.” 








THE RELIABILITY OF JUDGMENT OF PERSONAL 
TRAITS 


By JoHn Stawson, Division of Mental Defect and Delinquency, New 
York State Board of Charities 


Two outstanding features in the science of mental measure- 
ment clearly indicate the necessity for the standardization of 
judgment evaluation of personal traits: (1), an unfortunate 
lack of objective measures for human traits other than in- 
telligence, or educational attainment, so that evaluation of an 
individual in the shop, factory or school is still largely a matter 
of comparative judgment, such as ranking in order of merit; 
no contention is made that the judgment method is as desir- 
able as the standardized scale method employed in evaluating 
intelligence, but since the former is used of necessity, it must 
be raised to as objective a level as possible; (2), in the deriva- 
tion of a great many of our objective tests for intelligence and 
specific aptitude, we incorporate judgment evaluation in the 
composite criterion which determines the authenticity of these 
tests. 

In this paper the writer intends to give a brief report of the 
results of a study’ made in cooperation with the Bureau of 
Research of the Department of Education of the City of 
New York, the purpose of which was to determine the pos- 
sibilities of the order of merit method in evaluating personal 
traits of individuals engaged in a specific profession or work. 
Eleven personal traits were chosen as defined below, and inde- 
pendent ratings were made by 31 judges distributed among 
six schools (5+-5+-4+7 +6+4=31). 

In each of the six schools, twenty-five teachers known to 
all of the judges chosen in that school were rated in the eleven 
traits, which were selected because of (a), supposed impor- 
tance in the teaching profession, (b), competency of at least 
five rators to make judgments in them, and (c), distinctness 
or exclusiveness.* The judges were asked to rate the teachers 





1 The original report bearing the same title as this paper is on file 
at the Psychological Library of Columbia University, in manuscript 
form. 

2 Thorndike, E. L., “A Constant Error in Psychological Rating,” 
Jour. of App. Psychol., 1920, pp. 25-29, 
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in order of merit, i. e., placing the one who possessed most 

of the quality in question first and the one who possessed 

least of the quality last, by the use of the Stenquist “ Card- 
list” Combination Ranking Card,* which makes the task much 
less laborious to the rater and helps maintain uniformity in 
materials. Each judge, of course, made all ratings inde- 
pendently of the other judges. 

Trait Definition 

No. 1. APPEARANCE. (Personal neatness in dress, clean- 
liness, etc.). 

No. 2. TACT. (Ability to deal with others without giving 
offense ). 

No. 3. PUNCTUALITY. (Habit of being on time). 

No. 4. EFFORT. (How hard does this person try?). 

No. 5. JUDICIAL SENSE. _ (Fairmindedness, imparti- 
ality). 

No. 6. LEADERSHIP. (Ability to lead, guide, direct, in- 
fluence ). 

No. 7. COOPERATIVENESS. (Willingness to work ef- 
fectively with others). 

No. 8 PROFESSIONAL INTEREST AND GROWTH. 
(Interest in becoming a better teacher. Pro- 
fessional magazines, continued courses, lectures, 
associations, etc.) 

. UNDERSTANDING OF CHILDREN. (Insight 
into child nature. Success in handling children). 
. COUNTERACTING FACTORS. (This calls not 
for a judgment of the persons being ranked, but 
for a judgment of the environment in which each 
is working. Rank the person who is working un- 
der the greatest handicaps, No. 1, or first; the one 
working under the next greatest handicaps, 2d, 
etc.). Counteracting factors include such things 
as: especially difficult classes ; poor equipment ; de- 
pressing relations with other teachers or with 
supervising staff; bad hygienic conditions; poor 
health, etc—ALL AS COMPARED WITH 
OTHERS OF THE GROUP BEING JUDGED. 

No. 11. ALL-ROUND VALUE TO SERVICE. (As com- 
pared with all others. Not a total of the other 
items. A _ single estimate in toto disregarding 
specific items above). 





8 Stenquist, J. L., “An Improved Form of Rating by the Order of 
Merit Method.” Jour. Ed. Psychol., Dec. 1920, p. 526. 
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After an interval of about two weeks, the ratings were 
repeated by all of the judges independently of the first ratings. 
Sometime after the second set of ratings were made, the 
judges were asked to rate the same teachers using the same 
method as before, in acquaintance, i. e., placing the person 
best, known to the rator first, and the one least known last. 
The judges were also requested to answer a questionnaire, 
for the purpose of ascertaining the qualitative aspects of the 
data, such as the kind of criteria used in making ratings, etc. 
Statistical treatment of the first set of ratings the second set, 
the ratings in acquaintance and the answers to the question- 
naire enabled the writer to answer either provisionally or 
definitely the following questions : 

1. Which of the traits chosen lend themselves best to objec- 
tive evaluation? 

2. Do the traits tend to retain their positions for relative 
objectivity in different groups? 

3. To what degree can we reduce the differences in criteria 
employed? 

4. What is the effect of acquaintance upon variability of 
trustworthiness of judgment? 

5. What is the relation between judicial capacity and 
judicial consistency ? 

6. What is the effect of more than one trial of a rator’s 
judgment upon judicial agreement? 

7. What is the relation between judicial capacity and official 
position ? 

8. Does the distribution of judicial capacity tend to be 
specific or general? These questions will be dealt with sepa- 
rately in brief form. For a full account of statistical treat- 
ment employed and unabridged tables reference should be 
made to the original manuscript.* 

1. Relative Objectivity of Personal Traits. 

The degree of objectivity of a personal trait may be deter- 
mined by ascertaining the degree of group agreement, that is, 
the greater agreement there is among competent judges in 
assigning positions to subjects (independently of each other), 
the more objective is the trait.° Table I gives the average co- 
efficients of group agreement of the eleven personal traits 
arranged in order of magnitude. Each average coefficient 


of agreement (Av. frpq) is the average of the average co- 
4 Ibid. 


5 Hollingworth, H. L., “Experimental Studies in Judgment.” 
Archives of Psychol., No. 29, pp. 116, 118. 
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efficients of correlation between any judge, p, and any other 
judge, q, in each of the six groups of judges. In other words, 
each of the Av. rpq’s in Table I is the average of 68 inter- 
correlations. The P. E.’s are simply the variability coefficients 
for the six separate groups. The rpg, the coefficient of agree- 
ment for each of the six groups of judges, not shown in 
Table I, was obtained by the use of the following formula 
devised by Professor T. L. Kelley, which makes possible the 
computation of a group correlational index directly without 
the need of computing the individual correlations between 
the judges: 

M( i) 123(SX)? 

(M— Dm—ht M(M—1)n(n?—1) 


[pq is the same as Pearson’s r. 





Tpq — 1 


TABLE I 
Final * 
Posi- Av. fa0 P. E. 
tion Name of Trait (t-o) 
1 All-Round Value to Service................ 603 .032 
2 Cooperativeness Oe .013 
3  Leadership.... ridevad eeaie's pine ae .031 
4 Effort ATE .026 
5 Understanding of Children........... 472 . 036 
6 Professional Interest & Growth.... .. .. 470 .027 
7 Appearance... .. ey ORE L Ee .016 
8 Tact ke laheake .033 
9 Punctuality..... TT ee ee .040 
10 Judicial Sense.... Roe alate caherneas at .027 
11 Counteracting Factors. . Seat he Joe oa ee .038 


Final Positionjof Traits with their average coefficients of agreement 
and unreliabilities of these averages. 


Table I shows that “All-Round Value to Service” holds 
first place in objectivity; it has the highest coefficient of intra- 
group agreement. Judging from its P. E., its position in the 
series of eleven is relatively secure. It is to be noted that 
this is a complex or composite trait; the others are elemental 
or specific traits. This result, i. e., the greater objectivity of 
a composite trait as compared with specific traits is in accord 
with Wells’ findings in his study of literary merit. “Counter- 
acting Factors,” the condition that tends to lower the efficiency 
of a teacher, retains the last position with about as high a 





6 Wells, F. L., “ A Statistical Study of Literary Merit,” Archives of 
Psychol., No. 7, p. 14. 



















RELIABILITY OF JUDGMENT OF PERSONAL TRAITS 165 


reliability as “All-Round Value” retains first place. “Ap- 
pearance’ occupies a surprisingly low position. If we con- 
sider the fact that the correlation between two intelligence 
tests, one verbal and one non-verbal, is about .60, we can see 
the significance of the first three coefficients in Table I as 
regards objectivity. 

2. Intergroup Agreement as to Relative Objectivity. 

If we examine the differences between the coefficients to- 
gether with the P. E.’s in Table I, we can see at a glance that 
the relative positions of the traits are insecure. This is, of 
course, due to the lack of a high degree of agreement between 
the six schools as regards the relative objectivity of these 
personal traits. Table II illustrates this fact more clearly. 
Here we see that the relative positions of the traits obtained 
from the coefficient of agreement in each schoo! are by no 
means alike in each of the schools. Traits numbered 5, 7, 


~ 


TABLE II 
Trait Sch. Sch. Sch. Sch. Sch. Sch. 
No. Trait I mn ft iV VV 
1 Appearance. 9 7 4 4.5 8 6.5 
2 Tact . 6 3 7 10 = 10 3.5 
3 Punctuality. 2 2 g 9 9 10 
4 Effort.. 5 6 8 1 4 8 
5 Judicial Sense ss. .. 7 q 
6 Leadership.... 3.5 9 3 6 2 5 
7 Cooperativeness 3.5 5 1 4.5 5 3.5 
8 Prof. Int. and Growth... .10 8 2 7 3 6.5 
9 Understanding of Children...... 7 4 10 2.5 § 1 
10 Counteracting Factors...... nm & 6 . we oe 
11 All-Round Value.... - 1 5 a a | 2 








Relative positions of traits obtained from the coefficients of agreements 
in each school (1 indicates highest rpq and 11 indicates lowest rpq). 


10 and 11 hold their positions with a fair degree of consistency 
in that they at least stay in either the upper or lower half of 
the series of eleven. The positions of the rest of the traits 
are quite inconsistent. Trait No. 3, “ Punctuality,” occupying 
2d position in the first two schools and 9th and 10th positions 
in the last four schools, shows indications that a variable 
operating in Schools I and II, in a manner opposed to that 
in Schools III, IV, V and VI is tending to make this trait 
objective in the former case, and subjective in the latter. 

This disagreement with respect to objectivity among appar- 
ently homogeneous groups is of practical significance, for 


r 
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should the intergroup disagreement be inherent in rating by 
the order of merit method, local standardizations for each 
group would become necessary, thus greatly diminishing the 
usefulness of rating. The trait that would lend itself best to 
objective evaluation in one school of a large school system, 
or in one department of a large industrial organization, would 
not do so in another school or in another department. 

3. The Question of Criteria Employed. 

Although personal interpretation by the judges cannot and 
possibly should not be entirely eliminated, qualitative analysis 
of the questionnaire which was sent to the judges showed that 
standardization of the environmental conditions under which 
judgments are made and of the elements to be included in the 
interpretations will greatly reduce the disagreement and in- 
consistencies referred to above. The following are some of 


the more important points noted in regard to the standardiza- 
tion of criteria. 


A. Consulting Records: in schools I and II, practically all 
of the judges consulted records when rating for “ Punctu- 
ality,” but in the other four schools only two out of 21 judges 
consulted records. This explains the inconsistency referred 
to, which tended to increase intergroup disagreement, i. e., 
the trait occupied a high position for objectivity where records 
were consulted and a low one where they were not. This 
factor can of course be easily either eliminated or standard- 
ized. With the trait “ Professional Interest and Growth,” a 
somewhat similar situation occurred—records of professional 
courses and examinations taken were consulted by some judges 
and not by others. 


B. There was an indication that conditions were more 
favorable in some schools than in others for making ratings 
in a given trait. It became evident upon reading the question- 
naire answers that certain traits had greater judicial advan- 
tage in one school than in another, because one school placed 
greater emphasis upon the activities involving these traits 
than another school. In schools Il and VI where the trait 
“Tact” held a high position with respect to judicial agree- 
ment, we find statements characteristic of these schools only, 
such as, “ Most teachers in this school have some executive 
position which makes judgment in this trait fairly easy.” In 
the trait “ Effort,” some schools utilized as criteria extra- 
school activities, and others intra-school activities. ‘“‘ Counter- 
acting Factors” suffered in practically all schools due to lack 
of personal acquaintance with the conditions under which the 
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teachers were working, and it may be for this reason that it 
occupies a consistently low position for group agreement. 
The data points quite conclusively to the fact that ratings 
made in a trait which is brought more vividly or prominently 
to the attention of raters during an entire year are subject 
to less intra-group disagreement than ratings made in a trait 
rarely brought to the attention of the raters (or playing a 
relatively unimportant role in the life of that particular 
school ). 

C. It appears that simple definitions of traits should be 
supplemented by specific items wherever possible, in order 
to reduce variability in criteria. In “Appearance,” some 
allowed for financial condition of the persons rated; others 
emphasized how the wearing apparel suited the subject. 

4. Effect of Acquaintance. 

If we utilize the ratings made for acquaintance as hereto- 
fore described, and let 

Tig =the correlation between the rankings in merit given 

by judge I and the sum of the rankings given by the 
rest of the judges of his group excluding his own; 

=the correlation between the rankings in merit given 
by judge I and his rankings in acquaintance; 

=the correlation between the rankings in acquaintance 
given by judge I and the sum of the rankings in merit 
given by the rest of the judges of his group, excluding 
his own, 

we can partial out the factor of acquaintance and get 
Tig.a =the correlation between the rankings in merit given 

by judge I and the sum of the rankings given by 
the rest of the judges excluding his own, when 
acquaintance is kept constant (or when the judge 
in question is made equally acquainted with all the 
subjects rated). 

By this process we determined to what extent relative ac- 
quaintance with the subjects rated contributed to the judge’s 
agreement or disagreement with the rest of the judges. Does 
he deviate more from the group arrangement in rating the 
subjects whom he knows least, than in rating those subjects 
whom he knows most? It is impossible, due to lack of space, 
to give here the gains or losses when acquaintance is made 
constant for each of the 31 judges in each of the 11 traits. 
A summary is given in Table III by recording for each trait 
the average rig’, i. e., the average judicial agreement when 
the factor of acquaintance is untouched and the average 
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Y Ig.a’s', 1. €., the average judicial agreement when acquaintance 
is made constant. In the fourth column are given the dif- 
ferences as indicated, and in the fifth the P. E.’s of these 
differences. A positive difference indicates, of course, that 
acquaintance was a handicap or hindered judicial agreement, 
while a negative difference shows that relative acquaintance 
promoted judicial agreement, because when acquaintance was 
made constant (ruled out) judicial agreement was decreased. 


TABLE III 
Trait No. AV. Tig.a Av.tig.a Diff. (rig.a-rig) P.E. (diff.) 
. 636 + .034 .026 
. 583 +.021 .029 
.541 —.017 .034 
. 648 —.014 .031 
.445 —.049 .034 
. 583 —.036 .031 
.627 —.O15 .024 
; .574 —.007 .031 
C ; . 630 +.038 .036 
10 . . 385 —.015 .038 
11 . .651 —.010 .024 


Showing the influence of relative acquaintance in hindering or promot- 
ng judicial agreement in a given trait. 


OOonNDUI- Who 


It will be noticed that although there are 8 negative differ- 
ences, the unreliabilities are so large (exceeding the differ- 
ences in 7 out of the 11 traits, and in the remaining 4 being 
slightly less than the differences), we can fairly safely con- 
clude that the influence of the difference in relative degree 
of acquaintance on judicial agreement in situations such as 
were here encountered is negligible. 

This inability of acquaintance to definitely either raise or 
lower judicial agreement becomes evident upon considering 
the several ways in which this factor may operate. For al- 
though lack of acquaintance with one or several subjects may 
result in chance ratings, thus lowering the correlation between 
the unacquainted judge, and the rest of the group of judges, 
intimate acquaintance between a rator and subjects may also 
lower the correlation between the intimately acquainted judge 
and the rest of the judges, by the exercise of prejudice either 
due to friendship or to the discovery of peculiarities in the 
subject which are particularly abhorrent to the rater (and 
probably unknown to the less intimately acquainted judges). 
The positive and negative influences would then in the long 
run tend to balance each other, that is, acquaintance would 
have little or no effect. This finding is, of course, no argu- 
ment in favor of phrenology, because zero acquaintance was 
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eliminated before we started by choosing as judges those who 
had at least some acquaintance with the subjects. The dis- 
cussion refers only to relative degree of acquaintance and not 
to a total lack. This result should also not be confused with 
what we said about the advantage that the familiar trait has 
over the non-familiar one as regards group agreement or 
objectivity—“ trait acquaintance” is an important factor, but 
relative degree of personal acquaintance with subjects is not. 

5. Judicial Capacity and Judicial Consistency. 

Judicial capacity is determined by the degree of the judge’s 
agreement with the competent group of judges of which he 
isa member.’ Judicial consistency is determined by the degree 
of agreement between two or more independent ratings made 
by a judge.“ No demonstrable relation was found between 
the former and the latter. This relation was investigated 
with the intention of ascertaining whether it was possible 
determine capacity from consistency, since the latter can be 
determined so quickly and easily. The results were negative. 

The Effect of More Than One Trial. 

Contrary to expectations, we found that intercorrelating 
the average rank for two trials given by each judge, with the 
average rank for two trials given by each of the other judges 
did not increase group agreement or objectivity. As a matter 
of fact, the latter was lowered in some of the traits. In 


rating by order of merit, it seems that increasing the sampling 
of measures (ratings) does not bring us nearer the truth, 
i. €., increase group agreement. 
Judicial Capacity and Official Position. 
Table IV enables us to approximate roughly the relation 


TABLE IV 


Position for Judicial Total | Total for 

Capacity for lst | Pos. be- 

Official : 3 pos. low the 
Position ; : third 
A ; : 1 | No. &% 


Principals 4 | 25 2| 50 5| 4 oO 00 


Assis. Principals 10; 3 30. | 30 | 2|2 & 2 | 20 
12 et al Bled Sah all 





_ Showi ing the relation between offic ial | position ai nary judicial ¢ capacity. 


7 Hollingworth, H. L., op. cit. p. 109; “Vocational Psychology,” 
pp. 158, 159, 167 (New York, Appleton, 1916). 
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between official position and judicial capacity. In interpreting 
the table the percentages of principals, assistant principals and 
teachers occupying the first three positions in judicial capacity 
should be compared with the percentages occupying positions 
below the third. The superiority of the prinipals and assistant 
‘principals as judges is clearly seen. The assistant principals 
are better judges than the teachers, and the principals are better 
than the assistant principals. It is, of course, unsafe to make 
a positive conclusion from the small number of representa- 
tives that we have in each official position, but the indications 
are that executives and supervisors make better judges than 
associates or, stated differently, there is a positive correlation 
between official position and judicial capacity. 

8. Distribution of Judicial Capacity. 

The distribution of judicial capacity for each judge was 
investigated by plotting the number of times every judge 
occupied each designated position for judicial capacity in his 
group. In Fig. (a) below, we see that Judge A of School I, 
where there were 5 judges and, therefore, 5 possible positions 
for judicial capacity in each of the 11 traits, occupies the 
5th or last position in 10 out of the 11 traits. He is, there- 
fore, a generally poor judge; in Fig. (b) we see a generally 
good judge (but not so marked as the former). 





Fig. (a) Fig. (b) Fig. (c) 
Position-Judge A-School I Position-Judge C-Sch. II Position-Judge D-Sch. IV 














1st 1st sess Ist ° 
= - ee&eee _ **+ 
. * 
* -* 
= SESKEEEEEE = - ** 
6th . 


7th 


On the other hand, in Fig. (c) we see a judge occupying 
every possible position in his group (there were 7 judges in 
this school, and, therefore, 7 possible positions), from best to 
poorest. Figures (a) and (b) are illustrations of general 
judicial capacity, and (c) illustrates specific judicial capacity. 
If we plotted all the graphs here for each of the 31 judges, 
it would be seen that, roughly speaking, figures of type (a) 
and (b) are predominant, indicating that there is a greater 
tendency toward a general distribution of judicial capacity 
for each judge than toward a specific distribution. There is, 
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however, hardly strong enough evidence to conclude that if a 
person is a good judge in one trait he will be an equally good 
judge in another trait. There is an indication that he will. 
This relation is somewhat similar to the one found with intelli- 
gence—the preponderance of a general distribution of intelli- 
gence over a specific distribution. 














COMPUTING INTERCORRELATIONS OF TESTS ON 
THE ADDING MACHINE 


By Hersert A. Toops, Institute of Educational Research, Teachers 
College 


Any scale lately constructed for mental, educational or 
vocational measurement must have both high reliability and 
high validity. If it is to lay claim to any particular merit, 
it must correlate well with a valid objective criterion, or com- 
posite variable consisting of independent, socially derived and 
socially acceptable, measures of the ability being tested. High 
validity may be secured by proper selection and scaling of 
the elements of the individual tests composing the scale, and 
by weighting the individual tests in such manner that the 
weighted composite shall correlate most highly with the cri- 
terion. In this latter process the intercorrelations of a number 
of tests are often computed. This is a strenuous task unless 
systematic methods are used, since, if there are n variables 

n(n-1) 
including the criterion, there will be 





correlations to be 
2 

computed. It is the purpose of this article to outline a pro- 

cedure by which these can be rapidly calculated. 


The procedure is based upon the formula, 
N:2XY— 2X'‘= Y 
r= 
VN: EX*#—( X)? VN: 2Y*—@ Y)’ (1) 








in which X and Y are gross measures of transmuted scores, 
as described below. The speed possible with this formula 
depends upon: (a) the fact that these transmuted scores are 
numbers which are so small in magnitude that they may be 
readily multiplied mentally to obtain the XY products; (b) 
the use of a listing adding machine, the printed record from 
which supplies the required sums of Formula (1). The add- 
ing machine record affords a ready check upon the correctness 
of the work, and legibly prints all writing ordinarily required 
in making tally marks, raw scores, and the like, in other 
methods. 
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The steps in order are as follows: 


1. Tabulate the crude gross scores on the different tests 
in parallel columns, with a given subject’s scores all on the 
same horizontal row, A, Fig. 1. 

2. Determine the range of crude gross scores in each test. 
The range of a variable—=(highest score made in the variable 
minus the lowest score made in the variable plus 1). 

Now, for each test divide the range, thus determined, by 
20, and raise the quotient to the next higher integer. This 
quotient will be the class interval, I, which indicates for a 
given variable which column of the Standard Grouping Table 
(Table 1) is to be used in transmuting the crude gross scores 
to class scores. These transmuted class scores are the classes 
under which a given crude score would be tabulated if grouped 
and tabulated into not over twenty classes. This rule for 
determining I, examples of which are given in footnotes to 
Table 1* insures that there will be as many classes as possible 





1JIn practice, Table 1 is made out for the variables under considera- 
tion; that is, Table 1 fits only those tests where zero scores occur. 
Thus for Test 11, with no zero scores, but raw scores ranging from 
41 to 82, say, the total range is 42; and 42/20 gives I= 3. In this 
case, one would not use the I= 3 column of Table 1, but would con- 
struct a table: 

“Grouping Table for Test 11 only, I= 3. 


Transmuted Raw 
Score Score 

1 39-41 

2 42-44 

3 45-47 

4 48-50 






” 











As a desirable convention, the columns of raw scores should always 
begin with a multiple of the class interval as above and as in Table 1. 
The above table insures that the maximum number of classes will 
result. It is evident that the (I =3)-column of Table 1 cannot be used 
since it does not go high enough to contain the maximum score on 
é Test 11. 
Again, the crude scores on Test 19 range from 32 to 59. Here it 
would be possible to use l=3 column of Table 1, but this would give 
only ten classes. Here I =2, and the table to be constructed is: 












“Grouping Table for Test 19 only, I=2 


Transmuted Raw 
, Score Score 
f 1 32-33 
2 34-35 
3 36-37 

4 


38-39 
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with the restriction that there shall be not more than twenty 
classes. 
3. From Table 1, make out a cardboard stencil card (B. 
Fig. 1), of gross scores and transmuted classes for the vari- 
ables as shown. A shows only the first, three cases of the 
original data sheet; B shows only the first five of the total 
twenty lines of the stencil card; while C shows the trans- 
muted scores of the three names of A, and one additional 
name. Note that the column spacings of B and C correspond 


Crude Scores in Variable No. 





| 
« Name 1 


—_ 


oS \— = 


AlAloe, John..| 8 | 17 














3| | 
> |Beatty, Ray; 14 
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Transmuted Class Scores in Variable No. 


Name 











Aloe, John 5 
Beatty, Ray. 8 
Cole, Henry. 6 

7 


ypewritten 
muted Data 


s 


C 
Tran 


Dorsey, Tom 





Figure 1. Showing relative positions of Crude Data Sheet, Transmuta- 
tion Table, and Transmuted Data Sheet in obtaining the transmuted 
scores. A is the crude data, large scores. B is the transmutation table. 
in which the columns are taken from the standard grouping table accord- 
ing to the range of the variable in the respective columns. C' is the 
transmuted data sheet, on which the largest possible gross score is 20. 








176 TOOPS 


with A. In A, it is assumed that the range of variables is 
such that the class intervals are: 1,2; I,—=1; I,=3; L=1; 
[.—=2; 1,5; 1,2; and these columns are taken from the 
corresponding columns of Table 1 in constructing the stencil 
card transmutation table, B. 


4. Lay B upon A, allowing “Aloe, John” scores only to 
be visible. The transmuted scores are read from B, variable 
by variable, and either copied directly beneath on the trans- 
muted data sheet, C, or else read to an assistant who copies 
them onto the transmuted data sheet, C. Note that “Aloe, 
John” score of 8 on Variable 1, is found in the same column 
of B in the 8-9 group, and means a transmuted class score 
of 5; that his score of 4 on Variable 2, is found directly 
beneath on B to mean a transmuted class score of 5, etc. 

5. Have an assistant independently make the transmutation, 
and thus check the correctness of C. 

6. Typewrite C, making two carbon copies, in addition to 
the original typed copy, using relatively new carbon paper so 
that the carbon copies will be clear and distinct. Type the 
variable numbers as headings of the columns. After each 
third name allow an extra blank horizontal row. 


7. Proofread C, making corrections of any errors on the 
carbon copies also. 


8. Add up each of the several » columns on the listing 
adding machine imitating exactly the spacing of C, i. e., leav- 
ing a blank after every third entry. Label each column care- 
fully with the variable number. 


9. Check the work by placing the adding machine slip along- 
side the corresponding column of C. It will be found that 
the typewriter and adding machine, having the same size 
platens, or feed rolls, make the same spacings; hence corre- 
spording entries will be in opposition throughout and may be 
readily checked. The adding machine sum 5 at the feet of the 
columns, if the work has been correctly performed, will be 
the =X’s or the ZY’s of the formula, since the terms X and Y 
are merely relative. There will be » such columns. 


10. Mentally compute the squares of the entries of C and 
enter them into the adding machine in groups of three followed 
by aspace. There will be such columns. Label each column 
carefully with the variable number squared, thus, (1),*? (2),’ 
etc. 


11. Have an assistant check the squares after the manner 
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of procedure 9 above. The correct sums at the feet of the 
columns are the =X°*’s, or the SY*’s. 

12. Cut up both carbon copies of C into vertical columns, 
one variable to a column strip. 


13. Place carbon strip, m, against variables n-l, n-2, n-3, 
....l of original typed C in turn, obtaining for each an 
adding machine record of the XY products in groups of three 
entries and a space. Then place the (n-1) carbon strip against 
the n-Z, n-3, ....1 columns of C in turn in similar manner; 
and so on until every column has been paired with every other 
column. The multiplications can be done mentally in most 
cases. The largest possible product is 20x20—=400. A simple 
multiplication table, constructed by the author, enables one at 
a glance to note any desired product not remembered, without 
the inaccuracies attendant upon coordinate locating, which is 


ae ' n(n-1) 

the fault of most multiplication tables. There are — such 
column multiplications. Label each slip carefully with the 
product of the two variables concerned; thus, (1)x(2), 
(1)x(3), (1)x(4), ete. 


14. Have an assistant check the multiplications using the 
second carbon sheet strips. This work may proceed at the 
same time that the adding machine records are being made 

15. All the needed quantities are now known for substitut- 


oe . ; n(n-1) 
ing —> times in formula (1) to obtain the several — 5 


_ 


intercorrelations. This may probably best be done in syste- 
matic form as shown in Figure 2. In this table, 


15(a). The first entry of six in each large compartment 
is the SX Y product derived from the checked adding machine 
records of procedure 14, for the variables at the intersection 
of whose coordinates the compartment is located. Notice that 


these entries check for symmetry in rows and columns about 
the diagonals. 


15(b). The second entry of a large compartment is N-3XY. 
A calculating machine may be used to multiply all XY pro- 
duct-sums of the entire table in turn by the constant multi- 
plier, N. 

15(c). The third entry of a compartment is X-SY, for the 
compartment con idered; i. e., the 2X and ZY which are 
coordinates of the column and row respectively in which a 
given compartment is located. (The =X, SY, and =X? quanti- 
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15a. ) 


umns about the diagonals. 







FIG. 2 


N= 81 persons. 
tests. 














ties may be written into their appropriate places in procedure 
These products check for symmetry of rows and col- 


The Systematic Solution of Equation 1 in a Table of Intercorrela tions. 
Variable 1 is a criterion of general business ability; variables 9, 11 
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te Row 1 of each compartment is 5 XY 

’ Row 2 of each compartment is N- > XY 
Row 3 of each compartment is > X-> Y 
Row 4 of each compartment is N- > XY— | X- SY=—L 
Row 5 of each compartment isL- 1, ¥K 
Row 6 of each compartment is L-1/ yK7 °1/ 


Tha 


formula (4). 


sought in that compartment. 


*Note: Do not confuse this I (criterion variable) with I (class interval) of Table 1 and of 


15(d). The fourth entry of a large compartment is the 
f second entry minus the third, or N-SXY-2X:-SY; that is, the 
{ numerator of the correlation coefficient whose solution is being 


These remainders also check for 
symmetry of rows and columns about the diagonals. 
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15(e). The next step is the simplification of the denomina- 
tors, which is done in the last six horizontal rows of the table. 

(A). The first of these six entries is 2X? obtained from 
procedure 11. 

(B). The second row is N:3X?. 

(C). The third is (3X)*, obtained by squaring =X of 
the same column, found at the top of the page. 

(D). The fourth is (B)—(C); or, N3X?—(2X)?. 

(E). The fifth is the square root of (D). 

(F). The sixth is the reciprocal of ‘E), or 

1/V N:3X?— (3X)?. 
These reciprocals allow one to solve Formula 1, by a multi- 
plication process (where a calculating machine is available). 
Thus, r is equal to the simplified numerator (an integral 
number derived in 15d) divided by the square root of the 
539357 product of the simplified denominators of the two vari- 

ables concerned (derived in 15(e) (D); r is likewise equal 
to the simplified numerator, L, times the 15(e) (F) reciprocal 
of X times the 15(e) (F) reciprocal of Y. If we let: 


ee 


Sanenn | L=simplified integral numerator of 15(d) procedure ; 
H=simplified integral denominator expression of the X- 
variable in the 15(e) (D) procedure; 
K=simplified integral denominator expression of the Y- 


variable in the 15(e) (D) procedure. 

1/VH and 1/\/K=the results of 15(e) (F) procedure for 
the X and Y variables respectively, entering into a given cor- 
relation r xy. 


Then, r*¥ = . = (2) 


VH y 7K ‘VHI K 
in which L, and H and K and H-K are all integral quantities, 
the solution of the equation being readily performed on a slide 
rule. 
Or, by the second procedure, with a calculating machine, 


mn oe xi 
7" VH (/K 


in which L is an integral number and 
1 d 1 
VH * VK 
15(f). Assuming a calculating machine available so that 
the solution may be effected by Formula (3), the results of 
procedure 15(e) (F) are now to be written on the right hand 
side of the chart, noting that like reciprocals belong to the 
same ‘ariable numbers. (In all this work, it is to be remem- 





are decimal multipliers, as shown in Fig. 2. 
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bered that the terms X and Y are purely relative; that is, 
that in the correlation r,,, variable 1 is X and variable 2 is 
Y ; in the correlation r,,, variable 2 is X and variable 3 is Y; 
or that, finally, X is synonymous with “ variable of the two 
mentioned first” while Y is synonymous with “ variable of the 
two mentioned last.” It follows that =X,—=2ZY,, 2X,*=—=Y,’, 
H,==K.,, etc.) 

15(g). Now multiply the fourth entry of each large com- 
partment in each given row by the 15(e) (F) entry at the 
end of the respective rows. On a calculating machine this 
15(e) (F) entry, 1/\/K, is a constant multiplier for all com- 
partments of a given row. These products do not check for 
symmetry. : 

15(h). Multiply im columns, each fifth entry of a large com- 
partment by the 15(e) (F) entry, 1/\/H, at the foot of the 
respective columns. The result is r. This 1/VH entry is a 
constant multiplier for all compartments of a given column. 
The products must check for symmetry of rows and columns 
about the diagonals. 

With a calculating machine one may omit recording pro- 
cedures 15(b) and 15(c), and 15(e) (B) and 15(e) (C) 
and arrive directly at the integral quantities of procedures 
15(d) and 15(e) (D). If accuracy of r to not more than 
three decimal places is required, one may omit 15(e) (E) 
and 15(e) (F); in that case, the correlation coefficient is 
given by formula (2), which is solved by slide rule. 

It will be noted that 15(d) is the simplified numerator, in 
integral numbers; and that 15(e) (D) is the simplified de- 
nominator in integral numbers for a given correlation. 

In the work of procedures 13 and 14, the carbon slips may 
be held in proper place by thumb-tacking sheet C and carbon 
slip to a soft pine work board. It will be found that the 
grouping of records in groups of three’s will very greatly 
facilitate checking, eliminate checking errors, and relieve eye 
strain. 

The numerous checks for symmetry in the solution of equa- 
tion (1) in procedure 15 using Fig. 2 table, will be found to 
be a very adequate check on the solution of the equation. 
The final r’s, if the work is correctly done, will check for 
signs, decimals and absolute magnitude. The subtractions in 
procedure 15 (d) are algebraic; consequently r automati- 
cally comes out with the right sign if the work is done cor- 
rectly. 

With this method, the author’s best record thus far is the 
production of the XY product sums of 20 correlations of 151 
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cases each in an average time of 6 minutes 23 seconds each. 
The checking showed 3 entries of the 3020 XY entries to be 
in error. These were readily detected and corrected. It 
should be noted by way of caution that even a slight error 
in XY will make a large error in r; with reasonable care, 
and a possible rechecking of the adding machine slips, no 
errors should escape. The computation’ speed outweighs the 
consideration that a possible error will absolutely invalidate 
a given 7; statistical work should be accurate. The computa- 
tion of the XY products for N==50 persons will take a little 
more than 2 minutes per correlation; for N==75, a little more 
than 3 minutes, etc. Not so much time is required for check- 
ing; and finally, the systematic tabular form of solution of 
equation (1) is done more rapidly and with fewer errors than 
in any other method of solution; any errors in the solution 
of the equation will be located by the symmetry check as it 
is highly improbable that two errors will be made and of 
such an amount that the final r’s would check. If r,,# ry, 
and if r,, ¢r;,, the errors may be quickly detected. 

The greater the number of variables, the greater the saving 
in time effected by this method, as will be seen from the fol- 
lowing tabulation of the total number of adding machine 
records needed, and the number of adding machine records 


per intercorrelation for n=2, n=3, etc., variables, as n grows 
larger. 


TABLE 2 


Table Showing the Total Number of Adding Machine Records Re- 
quired, the Number of Intercorrelations Resulting, and the Num- 
ber of Records per Intercorrelation, as the Number of Variables, 
n, Increases 

Total No. of No. of Inter- No. of Records per 
Records correlations Intercorrelation 
=n?+ 3n =n(n—1) =n+ 3 


1 


is 
| 





COWUONAUI SW 


— 
| et et et et eet DD Cd 
| . «6 


Pitts: 
| &OANOOW 





| 
| 
| 
| 
| 





The fourteen adding machine printed records required for 
4 variables are shown in Table 3. Remember that the first 
four columns are adding machine duplications of the same 
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columns in Table C, which was previously derived from larger 
gross crude scores. None of the records of Table 3 are ever 
written down longhand, being instead always computed men- 
tally or by simple multiplication table up to products of 20x20, 
and entered directly into the adding machine. This gains 
speed over the slow writing of the human machine and secures 
legibility as well. All multiplications up to 15x15 may be 
easily memorized; beyond this point a simple multiplication 
table will refresh the memory for products up to 20x20. 
Every test worker knows most of the squares up to (20)?. 
It remains yet to be pointed out how the true averages and 
standard deviations may be derived. 
The standard deviation presents no problem. 
The formulae are: 


ox =IyV/ H= Ix VN2 X*—(¢z X)? (4) 
N N 








oy =ly VK =Iy VNE Y¥?— (CY) (4a) 
N N 
In which I is the class interval, determined in Procedure 2, 
and H and K have already been determined in Procedure 
15(e) (D). 
The averages present a little more difficulty. The formulae 
for the averages, in terms of transmuted class scores, are: 








Mp—= 2X ® 


N 
My = sY (5a) 


N 


These must now be located with reference to the raw scores. 
Suppose that by formula (5), Mx==3.47 when the class inter- 
val of variable X is, Ix==3. Our problem is thus to inter- 
polate, in Table 1 in the (I==3)—column, a transmuted class 
score of 3.47 into a raw score. Now the third transmuted 
class is 6-8 in terms of raw scores, the face value of the class 
being 7.0.* We thus know that the raw score average is 
somewhat more than 7.0; it is really 47/100 of a class interval 
more than 7.0. Now, (.47)(3)==1.41. And 7.0+1.41=8.41 
=M ,, in terms of raw scores. 

Again, if by formula (5a), M,=9.80, when I, =4; then, 
M,, in the terms of raw scores ==33.5-+- (.80)4=36.7. 
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The values of r obtained by this method are the identical 
values of r which would be derived from Fig. 1, C table, or 
from any exact method wherein the raw scores are grouped 
into coarser classes. By the procedure used in determining 
I, the group ig error of r, inherent in all grouping methods, 
will seldom, if ever, be more than one per cent of the magni- 
tude of the correlation coefficient, since the coarsest possible 
grouping, by the schema here presented, is in the case of a 
range of 21, in which the gross score of 20 becomes the 


TABLE 3 


The 14 Adding Machine Records Required in the Solution of the 6 Intercorrelations 
of 4 Variables. 
(Columns 1, 2, 3, 4 are copied directly from the transmuted class scores, Fig. 1, C. Each 
column of this table represents a separate adding machine record.) 




































































a ] l | 
Person Xi X: | Xs Xs Xi? X?? X3? | X? XiXe | XiX3 Xi 4 X2X3 | XX XX 
ral y 3} 2 a al g 4 | 1}; 3] 2 1 6 | 3 2 
K 3 2) S| 2 9 64) 25) 4) 6 | 15 | 6 10 | 4 10 
Cc} 8 12; 9 5) 64] 144 81) 2) 9% | 72 | 40 108 | 60 | 45 
_ a Oe | | 
D 4 3 6 8 16 9) 36) o&) 12 | mw | 3 18 | 4 | 48 
E| 6 7 8 8 36 49| 64| 64| 42 | 48 | 48 | 56 | 56 | 64 
Fi 9 8 19 9} 81) 64| 100; 81| 72 | 90 | 81 | 80 | 72 | 90 
| | | | 
G| 12) 10 8 6 144/100) 64| 36/| 120 96 | 72 | 90 | 60 48 
H | 10| 11) 12} 10) 100| 121 | 144 | 100 | 110 | 120 | 100 | 132 | 110 | 120 
I | 13] 12) 10) 9| 169) 144 | 100| 81 | 156 | 130 | 117 | 120 / 108 | 90 
} } } 
I} 8 6 10 11 64| 36 | 100 121} 48 | 80 | 88 | 6 | 66 | 110 
$$ $—_—|—_}|——_} —_}|—_| —_-| ___ ——| 
Sum | 74) 74} 80| 69| 684| 680 | 718 | 577 | 665 677 | 685. | 670 | 563 627 
|\—|— | — | — | —— | —__ |} | | SE | 
Nota- | | | | | 
tion |X, =X, at X4|= Xz X2?| 2X3? 2XP |= XiXs) aie os XaXe|Z XoKayS XaX4) 2 XsXo 
' | | } | | 








eleventh class. Since the correction, due to coarseness of 
grouping, becomes negligible if at least twelve classes are used, 
it will be seen that the denominator, 20, used in determining 
the class intervals, is a happy compromise between ease of 
mental multiplication and requirements for disregarding the 
grouping error involved by this method. The grouping errors 
in the use of this method may be entirely disregarded. 


Nore.—There are two current usages with respect to the face value 
of a class, 6-8. The first is that advocated by Pearson, involving the 
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determination that a score, 6, is closer to 6 than to any other value. 
Consequently the class, 6-8, will include all gross scores of 5.500 up 
to and inclusive of 8.499; whereupon the face value of the class is 
5.500 + 8.499 7.0. The second usage, growing out of the test con- 
2 
sideration that one cannot obtain a test score of 6 unless his ability is 
at least 6, considers the class, 6-8, to extend from 6.000 to 8.999, where- 
upon the face value of the class is 6.000-+ 8.999=7.5. It is essential 


? 

to note that people advocating the latter procedure, when finding the 
average of a number of test scores without grouping are likely to add 
all together on the adding machine and divide by N without thinking 
to add the correction of %; when this error is made, the implicit as- 
sumption is of course that a score of 6 is nearer 6 than any other 
figure, and this procedure is the first method referred to above. Con- 
sequently many errors occur in the second method. Inasmuch as the 
test units are not absolute measures from a stationary zero, such as 
the units of a yardstick, no possible good can come from adding the 
Y% gross score correction, and confusion is the only likely outcome of 
the introduction of a procedure which is not universally applicable to 
statistics in other fields of research. 

As a desirable convention, the lowest class of a contingency table 
might always be given in the form of the extreme limits classifiable in 
that class, after the fashion of some of the English writers. The face 
value of any given class can always be easily determined by taking 
one-half of the sum of its two extreme limits. 
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A GLIMPSE OF HIGH SCHOOL COURSES AS 
MEASURED BY THE OTIS TEST 


By RutH Swan Crark, Ph. D., Bureau of Vocational Service for 
Juniors, New York City 


Feeling the demands of modern industrial and social condi- 
tions, High Schools of today are offering special courses of 
study in order to train their students for various kinds of 
vocations. It is important to study, not only the subjects 
taught in these different courses to meet these industrial 
demands, but also to study the students who voluntarily choose 
to take these courses, for the purpose of discovering whether 
or not they show distinguishing mental characteristics. 

it is with this problem in mind that the present survey has 
been made. The Otis Group Intelligence Test, Advanced, was 
given to students in both the entering and graduating classes 
of various High School Courses in New York City. In this 
way, studies were made not only of the students who volun- 
tarily selected these courses, but also of those who were suc- 
cessful in meeting the demands of the courses, so that they 
were allowed to graduate. The High School courses which 
were chosen for this survey were the Academic, the Com- 
mercial, the Technical, the Industrial Arts, and the Dress- 
making. 

The outstanding mental traits of the High School student 
of the Academic Course, so far as these could be detected 
by the use of the group intelligence test, are given in the fol- 
lowing table : 

The above facts indicate that the students who voluntarily 
chose the High School Academic Course were those of a gen- 
eral intelligence above the average for boys and girls of their 
own age. The range of intelligence within the group, how- 
ever, was broad, so that one out of every 20 of these students 
were below the average, while nearly two-thirds of the whole 
class were above the average. These students were, more- 
over, accurate workers, having more than two-thirds of the 
work that they attempted correct. They also showed the 
ability to readily solve problems dealing with concrete situa- 
tions. This ability was outstanding as compared with their 
memory for words in logical sequence. 











CLARK 


TABLE 1 


High School Academic Course. (Boys and Girls) 


lst Term 8th Term 
IRS ho Sai 6.b.0b modded cee aear 304 151 
I reais Ais bo G 44:16.h dns eeu o'e.e eaara se 129.9 
7Av. Physical Age ..... samba Seta 14-2 
5Av. Mental Age . Pek Oe 16-4 
*Av. Intelligence Quotient oKenba 115.2 
Middle 50% Range of I. Q.. ee 
Speed (% of Questions Answered).......... 75. 7% 
Accuracy (% of Questions wee ere > , 
‘Distinguishing Best Test . ........... Arithmetic Analogies 
Distinguishing Poorest Test.. ......... Memory Memory 
No. of Students with Intelligence 

above average (I1.Q.+110). ints: V7 79.4% 
No. of Students with weep below 
Average (I.Q. -90).. sae wea 66% 


The graduates of this course were of similar intelligence. 
The group was, however, more homogeneous; those with in- 
telligence below the average were not with those who had com- 
pleted the demands of the course. In fact, four-fifths of the 
class were of intelligenc above the average for those of their 
age. The graduates were also more accurate workers, as 
compared with the entering students. Their memories again 


showed poorly in comparison with their other mental traits. 
In the case of the graduates, however, their ability to perceive 
relationships was superior to their ability to solve problems 
of a concrete nature. In general it may be said that this aca- 
demic course adapted itself to students of a general intelli- 
gence above the average. A student with intelligence below 
the average would have but one chance in 150 to graduate, 
while the student of average intelligence would have one 
chance in 5. The correlation between the intelligence quot- 
ients of these graduates and the average of their school marks 





1 This number represents the total number of students enrolled in 
the course. 

2 The highest physical age credited to a student was 16 years. 

8 This average was secured from the average score of the group 
translated into terms of the Binet Mental Age, as given in Otis Manual 
of Directions, 1921. 

4 This quotient was secured-by dividing the average mental age by 
the average physical age. 

5 The “ Best” and “ Poorest” Tests were determined by a study of 
the number of errors as compared with the number of right answers 
in each test of each student. These tests were then arranged in order 
of their ease and difficulty for all students in all courses. Any devia- 
tion in this order shown by the students in any one course was con- 
sidered a “ distinguishing best” or “ poorest” test. 
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for the four years was .43 (.04 P. E.) (Pearson). This is 
not a high correlation, but in relation to similar correlations 
with the students in the other courses, it is the highest of all." 

A similar study of the students of the Commercial Course 
shows the following facts: 


TABLE II 


High School Commercial Course. (Boys and Girls) 
[Ist Term *6th Term 


No. OF DRIER. 5 nc ccc csccs eh ats Selituie 287 102 
a ee rae 119.1 147.9 
REI 1200 82 < 228 e 2 rece en 14-2 15-11 
Av. Mental Age Ne a tea tins ter oa he WAX 15-7 17-5 
Av. Intelligence Quotient. . de Bae we 110. 109.4 
Middle 50% Range of I. Q.. ere 103-114 
Speed (% of Questions Answered). ee 80.7% 
Accuracy (% of Questions Correct)........ . 64.4% 75.1% 
Distinguishing Best Test. . Sere Proverbs 
Distinguishing Poorest ics wn ac nnus Fol. Simi- 
Directions larities 

No. of Students with ee above 

Average (1.Q. +110 43.2% 45% 
No. of Students with Intligencet below w Average 

Le ae 11.4% 2.9% 


From the above facts, it appears that those who voluntarily 
chose the Commercial Course were students of high average 
intelligence. They were rapid in their work without being 
very accurate. They showed a better memory for words in 
logical sequence than the ability to adapt themselves to the 
following of directions of a specific nature. Those who gradu- 
ated from this course, on the other hand, were quicker, more 
accurate, and more intelligent. It is true that the average 
intelligence of the graduating class was the same as that of 
the entering class, but there were fewer of those with average 
and less than average intelligence amongst the graduates than 
amongst those who were entering the course. It is interesting 
to note also that the graduates of this course showed more 
conspicuously their ability to grasp and to deal with meanings 
of words, as in the Proverbs Test, than they did their ability 
in memory: the trait which characterized the entering group. 
The correlation between the intelligence quotients of the 
graduates in this course and their average marks for their 
three years of work was .34 (.06 P. E.). This seems to indi- 





1Cf. Proctor, W. M. Psych. Tests and Guidance of H. S. Pupils. 
Jour. of Ed. Research, Mon. I, 1921, p. 16. 
2 This course was for three years only. 
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cate that not as much dependence is placed in this course 
upon general intelligence, as in the Academic Course.’ 

The following table shows similar mental traits for the stu- 
dents in a Technical High School Course: 


TABLE III 
Technical High School Course. (Boys) 


lst Term 8th Term 
ee 100 
oo cia kia cs 8 ekbeulv ees 121.9 


Av. Physical Age. . voettesceeteeetnee 14-3 
Av. Mental Age. ee eee 15-8 


Av. Intelligence Quotient... TLS Sen ee 110. 

Middle 50% Range of I. Q... ea 

Speed (% % of Questions Answered). . tke 

Accuracy (% If Questions Correct)........... _70.2% 

Distinguishing Best Test Opposites Arithmetic 
Distinguishing Poorest Test................. Geometric Similari- 


Figures ties. 
No. of Students with  ;rae above 


Average (1.Q. +110 wee. 6=5$.9G% 44.4% 
No. of Students with Inteligence below 
Average (1.Q. -90).. pada 10.5% 2.% 


The boys who were just iediaiee this technical course seem 
to have been those of high average intelligence, not as quick 
in answering the questions as those of the Academic and 
Commercial Courses, but resembling the students of the Aca- 
demic Course in their high degree of accuracy. They were 
conspicuous in their ability to note logical relations, as in the 
Opposites Test, but they less readily adapted themselves to 
following the detailed directions of the Geometric Figure Test. 
The graduates of this course were of similar intelligence, but 
were quicker and more accurate in their work. They showed 
themselves better able to deal with arithmetic problems, which 
were concerned with concrete situations, than with the form- 
ing of abstractions, which were called fo: in the Similarities 
Test. The correlation between the intelligence quotients of 
these graduates and their average school marks for the four 
years was .34 (.07 P. E.). This correlation resembles that 
found for the students of the Commercial Course more than 
that found for those of the Academic Course. 





1Cf. Bock, W. F., Intelligence of High School Seniors, p. 145. 
Madsen, I. N., Group Intelligence Tests as a means of prognosis in 
High School. J. of Ed. Research, III., pp. 43-52; 396-398. 

"Not all the entering and graduating students enrolled in these classes 
were tested. Instead, 100 students taken at random from each group 
were used to represent them. 
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In Table IV are found the measurement of the same traits 
for the students of an Industrial Arts Course. 


TABLE IV 
High School Industrial Arts Course. «(Girls) 


lst Term 6th Term 


No. of Students............... tins 110 34 
Av. Otis Score......... 5 reread teeta aa 132.7 
Av. Physical Age. . ee Be orca 14-8 16-0 
Av. Mental Age. ree ae ee ee 16-0 16-6 
Av. Intelligence Quotient... . 5g So mia cen ie 103.1 
Middle 50% Range of 1.Q....... teeeaeee 99-117 97-108 
Speed (% of Questions Answered). Sy 74.7% 75.6% 
Accuracy (% of Questions Correct) . 68.9% 71.9% 
Distinguishing Best Test........ é; 5 . Memory Proverbs 
Distinguishing Poorest Test... .. a Analogies Arithmetic 
No. of Students with Intelligence. Be dition 
above Average (I.Q. +110) 449, 20.5% 

No. of Students with Intelligence t below 

Average (1.Q. —90).... ae 6.4% 8.8% 


The students entering this Industrial Arts Course resemble 
more the students of the Commercial Course than those of the 
other courses. They are of high average intelligence, quick 
at their work without being especially accurate, and showing 
an ability in their memory for words in logical sequence. Their 
“ poorest ” test, however, was different. The students of the 
Industrial Arts course were less apt in perceiving relationships 
as such (Analogies Test) than they were to follow specific 
directions, which was the greater difficulty for the Commercial 
Course students. 

In comparison with the students just beginning the Indus- 
trial Arts Course, the graduates of this course showed a simi- 
lar speed in their work, but they were a little more accurate. 
As a group, they showed less general intelligence. This may 
be due to the fact that more intelligent students find the course 
progressing too slowly for them, or it may mean that the 
course adapts itself more to students with a special ability in 
art than to students with merely a general intelligence. The 
correlation between the intelligence quotients of these gradu- 
ates and the average of their school marks, .36 (.10 P. E.) 
shows as close a relationship between intelligence and success 
in the course as has been shown in both the Commercial and 
Technical Courses. However, the size of the Probable Error 
of the correlation shows its unreliability. 
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The mental traits of the students of the Dressmaking Course 
are given in the following table: 


TABLE V. 
High School Dressmaking Course. (Girls) 


lst Term 6th Term 
No. of Students. 110 51 
Av. Otis Score...... 109.6 119.8 
Av. Physical Age. . 14-7 16-0 
Av. Mental Age. 14-9 15-7 
Av. Intelligence Quotient ‘ 101.1 97.4 
Middle 50% Range of 1.Q 91-119 91-102 
Speed (% % of Questions Answered). . .. 70.5% 69.4% 
peal (% of Questions Correct) 63.6% 70.7% 
Distinguishing  « Following Arithmetic 
, Directions 
Distinguishing Poorest Test. ae Geometric Analogies 
Figures 

No. of eer of Intelligence above Average 

(1.Q. + 26.3% 11.7% 
No. of he ‘aa Intelligence below Average 

(1.Q. -90).. Peeiea rete .. 22.7% 21.5% 


The girls voluntarily choosing this course in dressmaking 
were those of average intelligence. They were not as quick 
as the Commercial Course students but they showed the same 


degree of accuracy. Unlike the Commercial Course student, 
however, they displayed more ability in following directions 
accurately. They resembled the Technical Course students 
in the difficulty they found with the Geometrical Figure Test. 

The graduates of this course were also of average intelli- 
gence. This group was, however, a more homogeneous one, 
for the entering students included many students of intelli- 
gence above the average for those of their age, while these 
did not appear amongst the graduates to such an extent. Again 
one might conclude that the course did not progress fast 
enough for students of higher general intelligence, or that the 
course adapted itself more to students with a special ability 
in hand work than to students with merely a general intelli- 
gence. The correlation between the intelligence quotients of 
these graduates and the average of their school marks for the 
three years was .35 (.06 P. E.). This correlation indicates 
as close a relationship between intelligence and success in the 
course as has been pointed out in the Commercial, the Tech- 
nical, and the Industrial Arts Courses. Selection within these 
courses, therefore, does not take place on the basis of intelli- 
gence, as much as is apparent in the Academic Course, but 
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seems to depend upon other factors, such as ability in hand 
work. 

In conclusion one may state that the High School Academic 
Course adapts itself to students with intelligence above the 
average for those of their age. Moreover, selection within 
the course seems to be based upon the possession of general 
intelligence. On the other hand, while special courses, such 
as the Commercial, the Technical, the Industrial Arts, and the 
Dressmaking require general intelligence of an average and 
high average degree, selection within these courses does not 
seem to be based upon this factor alone. In general, it is 
apparent that High Schools are not adapting their courses to 
train students of a general intelligence below the average. 
Such a student has one chance in 150 to graduate from an 
Academic Course, but three times as many chances to gradu- 
ate from a Technical Course and five times as many chances 
to graduate from a Commercial Course. 

No attempt has been made in this survey to study all the 
different High School Courses, nor yet to claim that these 
findings will apply to schools in other communities. The 
purpose of this study was to discover whether or not group 
differences existed from a psychological point of view. It is 
apparent from the above results that such differences do 
exist and that some of these may be detected by the use of a 
group intelligence test. With such differences studied and 
measured, school authorities will find them of value in the 
educational and vocational guidance of the students who 
naively choose to take special High School courses. 





THE TESTIMONIAL AS AN ADVERTISING APPEAL 


By Ecpert M. Turner, College of the City of New York 


a The purpose of this paper is\to attempt to determine the 
value of the testimonial method of advertising from the yiew- 
point of the permanency..of the testimonial writers’ interest 

4n.the worth of the article. The 
paper does not attempt to analyze the number of sales made 


by such appeals, but attempts to, determine whether or not the 
. writer of the testimonia } 


1. Continues to believe in the worth of the article he first 
recommended. 

2. Whether he bothers to answer the letters of those re- 
questing advice. 

3. Whether or not men and women differ with respect to 
the ways in which they as testimonial writers react toward 
letters written to them for advice. 

4. Whether or not people with the medical degree differ 
from an unselected group in the way in which they as testi- 
monial writers for an article of a medical nature react toward 
letters written to them for advice. 

A business concern may, without decreasing its sales, spend 
thousands of dollars yearly for electrical signs which amuse 
and interest spectators, who often are unable to tell the name 
of the article being advertised a few minutes after enjoying 
the dazzling lights. However, a concern depending mainly 
upon the testimonial method of appeal cannot afford to invest 
money in printing and circulating these testimonials unless it 
is certain that these appeals are not actually interfering with 
and decreasing its possible sales. 

Many advertising appeals do not lead to an immediate 
response, but require deliberation, comparison and choice. 
Professor H. L. Hollingworth (Advertising and Selling), 
calls such advertising appeals which bring into activity the 
higher thought processes, such as reasoning and decision, the 
¥‘ long circuit appeal.” He calls those advertisements which 
lead to almost immediate response, because of their appeal to 
some fundamental interest, feeling or instinctive tendency, the 
Y short circuit appeal.” 

Analysis of the method, of advertising the article studied 
in this paper indicates that/the appeal is a combination of the 
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short and long circuit appeals.) (The article being of a medical 
nature immediately appeals to those suffering the ailments it 
claims to cure. This short circuit appeal thus causes the 
observer of the testimonial pamphlet cover to read its contents. 
From this point on the short circuit appeal is overshadowed 
by the long circuit appeal, i. e., argument, reasoning and 
decision.} If decision is not made, either t6 buy or not to buy, 
the reader often writes to one of the testimonial writers for 
advice,—thus protracting the appeal. Should the answer fail 
to come, or be unfavorable toward the article, the appeal is 
probably lost, and from the viewpoint of the concern a sale 
is lost,—hence the need of investigating the worth of the 
testimonial as an advertising appeal. 

The business house for whom this study was made has 279 
testimonials in eight different pamphlets. These pamphlets 
are sent to people whose names are secured in the usual mail 
order house method, and also to those inquiring directly about 
the article. Of the 279 testimonials 53 were written before 
the year 1916, 85 in the year 1916, 97 in the year 1917 and 
44 during and since the year 1918. 

In making this study the same type of letter was used in 
approaching the testimonial writers, stating that the writer 
had noted that he or she was an enthusiastic user of the article 
advertised and would appreciate their advice about it as he 
was contemplating the buying of it. A stamped envelope 
was enclosed in all of the letters, and “ Mr.” signed on all 
letters addressed to men and “ Miss” on all letters to women. 
The reason for this precaution was that a sensitive person 
might object to write to one of the opposite sex about a} 
medical article. 

The replies and returned letters were classified as: 

1. Good—namely, all those replies recommending the 
article, and which might influence one to buy it. 

2. Poor,—those which would cause one not to buy the 
article. 

3. Returned, 
to locate. 








all letters returned by Post Office as unable 


4. No answer,—all letters which brought no response and 
were not returned. 

5. Deceased,—letters returned advising that person ad- 
dressed had deceased. 

The limitations of this classification are evident. The 
replies classified as “ Good ” ranged in their advice all the way 
from a mere approval of the article to an enthusiastic “all 
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power ” belief in it. Likewise, the letters classed as “ Poor” 
ranged from a mere disapproval to a strong belief that the 
article was detrimental and dangerous to health. / The letters 
were judged by four people,—the Medical Advisor to the Insti- 
tution, the President, the General Manager and the Manager 
of the Mail Order Department. For the most part, their opin- 
ions were unanimous. Undoubtedly a ranking on the basis of 
1 to 5 would have been more desirable, but it is extremely 
difficult to get business people to use and think in fine measures 
of gradation. Naturally those letters classified under “ Good ” 
were the only ones which could be considered as making a 
positive appeal, while those classified as “ Returned,” “ No 
answer ” and “ Deceased ” certainly did nothing to encourage 
the inquirer to buy, while those classified as “ Poor” would 
tend to cause the inquirer not to buy. 

The following table gives a statistical analysis of the returns 
for the 279 letters written: 


TABLE 1 


SUMMARY OF REPLIES AND RETURNE® LETTERS. 

































































| Men, i Men, Women and “Institutions 
Women | | \Inst - | During 
\& Insti- Men | Women; tions | Before Year Year &after 
tutions | |} 1916 1916 1917 1918 
No| % |No! &% 'No| % |No}| % [No | % INo Jo No| % No | % 
| | | —— | —- | — | S|! — | — | - ——| —__ 
Good 138/49. 4) 90} 47. 9} 46/56.1) 2/22. 2| 17/32.1) 39)45 9] 56)57. 7| 26/59 1 
Poor 19| 6.7/ 9| 4.8} 8| 9.8) 2\22.2| 611.5] 5| 5.9| 6) 6.2 2) 4.5 
eaiscanmamean Wane Wniehet SPe: etek citi: TER: uaa Cate! We: se: See Oty 
Returned 52/18 6| 40/21 3 iE 4) 1/11.2| 13/24.5| “2124 7 1111 3 7/15.9 
——__— -—- §-— - 1 —-— — ——-_ | -+-—— — —} _ —————— — ———— 
No answer.| 55/19. 7! 35 |18. 6 16) 19.5) 4/44 4) “13/24 5] 15117.6 19/19.6| 8/18.2 
—— S§= [ine ee | ee | ee | me § <oe ;-—-|——- ——— ————— 
Decreased.| 15| 5.6) 14| 7a 1] 1.2 4| 7.6 5) 5.9) 5) 5.2) 1) 2.3 
Total 279) 2) | 9| + +|s3) |es||9o7| | aa 


1188) 82| 9} | 53} | 85) | 97 a 











It will be noted that only 149.4% of the replies were favor- 
able; that is, about one-half of the letters written to the orig- 
inal testimonial writers elicited favorable replies. Of the 
replies, 6.7% were of the nature which would discourage the 
potential buyer. The remaining 43.9% of letters failed to 
bring forth a response and thus made no selling appeal what- 
ever.f If anything like these percentages hold good in other 
concerns the testimonial method of appeal is very wasteful 
since it favorably impresses only about one-half of the pros- 
pective buyers who have actually shown an interest in the 
article to the extent 6f writing for advice about it/ 
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/Furthermore, it will be noted that the more recently written 
testimonials are more effective than the older ones./ Of those 
writing testimonials before the year 1916 only 32.1% replied 
favorably; of those writing during the year 1916, 45.9% 
replied favorably; of those writing during the year 1917, 
57.7% replied favorably ; during and since 1918, 59.1% replied 
favorably. The favorable responses for the different periods 
range from 32.1% to 59.1% /This calls attention to the im- 
perative need of continually revising testimonial pamphlets, 
but even the sa percent of 59.1 seems to the writer to be 


amazingly low. 

ro speak with certainty of this apparent wastefulness it 
would be necessary to compare the sales resulting from the 
testimonial appeal with the sales resulting from an appeal 
free of testimonials. /This study might be conducted by send- 
ing two unselected groups two different forms of appeal. An 
analysis of the number of sales would then indicate the rela- 
tive values of these two methods, and also whether or not the 
fact that from 40 to 68% of testimonial writers fail to respond 
to inquiries about the article they have recommended decreases 
the sales below the number made by the other appeal. It is 
believed by some that people prefer to write testimonial writ- 
ers instead of the business concern, feeling that the advice 
will not be biased, and also that they escape the many follow-up 
letters sent by the usual mail order houses. Should the 
analysis indicate the value of the testimonial method it would 
then call attention to the need of devising a way of increasing 
the percentage of favorable replies. 

It is of value to determine whether or not there are any 
characteristic differences between men and women with re- 
spect to the ways in which they behave as writers of testi- 
monials. / Of the 279 testimonials 188, or 67.4% were written 
by men; 82, or 29.4%, by women and 9, or 3.2%, by repre- 
sentatives of institutions. The concern claims that there was 
no conscious selection of men’s testimonials, but that when 
the pamphlets were compiled the good testimonials were 
selected regardless of sex. It may be that women are some- 
what more reserved than men about recommending an article 
of a medical nature. | While the concern has no record of its 
sales according to sex still the general belief is that more men 
than women buy it. 

It will be further noted that 47.9% of the men and 56.1% 
of the women answered the letter, requesting advice, favor- 
ably. That is, 8.2% more women answered the letter favor- 
ably than men. On the other hand, it will be noted that 4.8 
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of the men wrote unfavorable replies and 9.8 of the women 
wrote unfavorable replies. That is, 4% more women wrote 
unfavorable replies than men. {While these differences are 
small still the indications are that a greater percent of women 
wrote both favorable and unfavorable replies than men. 

Among the testimonial writers were a few doctors, but the 
number of men and women holding the medical degree is too 
small to permit any valuable generalization, still the table 
below shows indications that are worth considering: 


TABLE 2 
COMPARISON OF REPLIES FROM DOCTORS AND NON PROFESSIONALS 


Doctors Non-Professionals 











see 


Returned... . 


























It will be noted that of the 18 doctors who first recom- 
mended the article only 44.4% wrote favorable replies to those 
requesting their advice, as compared with 49.8% of 261 people 
other than doctors, who answered favorably. Also that 22.2% 
of the unfavorable replies were made by doctors as opposed 
to 5.8% gf the unfavorable replies written by others than 
doctors. /This seems to indicate that from the viewpoint of a 
business concern those of the medical profession are less de- 
pendable gs testimonial writers regarding articles of a medical 
nature. iA 

CoNCLUSIONS 

1. Approximately an average of only about 50% of testi- 
monial writers answer favorably in response to inquiries from 
prospective buyers of the article in question. 

2. About 6 or 7% of testimonial writers give unfavorable 
and unsatisfactory replies in regard to an article they at first 
recommended. 

3. Recent writers of testimonials are inclined to write more 
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favorable replies,—hence the need of frequently revising testi- 
monial pamphlets. 

4. The difference between men and women with respect 
to the ways in which they behave as writers of testimonials 
is small, but the indications are that women write a greater 
percentage of both favorable and unfavorable replies with 
reference to testimonials. 

5. From viewpoint of a business concern, those of the med- 
ical profession are less dependable as testimonial writers rela- 
tive to an article of a medical nature than unselected people. 





EFFECT OF LETTERS AND SYLLABLES IN 
PUBLICITY 


By Epwarp S. Jones, Oberlin College 


It is patent from a casual notice of trade names and other 
symbols used in advertising that certain forms are much more 
desirable than others. Comment regarding the characteristics 
of letter and syllable forms in language go back at least to the 
rhetoric of Dionysius of Hallicarnassus who undertook to 
analyze the affective quality of various Greek letters. Un- 
fortunately a study of this sort for one language does not 
carry over into other languages well, as many letters are pro- 
nounced somewhat differently in different tongues which may 
have derived their forms from the same roots. 

Some one has suggested that the affective quality of articu- 
lated forms is closely related to, if not contingent upon, the 
affective quality of predominant tastes in that part of the 
mouth effecting the articulation. The harsher forms, for 
example, such as “g” involve the back of the tongue ande 
consequently are connected with the sensation of bitterness 
assuming an unpleasant flavor. It is presumed that early in 
the language experience of the child, when the taste sensa- 
tions are in relatively pure form, these affective conditions 
are started. Words denoting disgust, and the emotion of fear 
(including certain religious utterances) are cited as evidence. 

Of course it is possible that other influences are much more 
potent in such affective tendencies (which are really condi- 
tioned reflexes) than the sensations of taste. This is specially 
probable in consideration of the recent agreement on the part 
of investigators that taste “ localization” on the tongue is not 
nearly so definite as it was once thought to be. Letters in- 
volving extensive movements of the uvula or throat, such as 
“k,” “g” and “q” are utilizing the same muscles which 
are active in sudden emotion, particularly despair. The 
sonants “b,"\“d,” “g” and “v,” which involve a vibrating 
larynx as well as throat movements may be specially correlated 
with sustained excitement. The “1,” “r,” and “s” forms 
seem to include front of mouth movements, and probably a 
tendency to widen the mouth, and may thus be correlated 
with the emotions of love,—hence engender pleasure. The 
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““s” form, however, is considered quite unpleasant by Diony- 
sius on account of the accompanying hissing sound, which 
should remind us that auditory imagery must of course com- 
plicate any effort to correlate feeling quality with the articula- 
tion peculiar to language forms. 

But this is all speculative. If a conmection between our 
simple letter forms or syllables and affective states does exist, 
it should be subject to investigation. No doubt many authors 
have an undSual sensitivity for the “feeling” of forms in 
prose or poetry. It is possible also that experienced adver- 
tisers are able to select symbols which are calculated to be 
not only pleasant but also easily remembered as compared to 
other forms. Such differences in affective and memory 
“value ” in letters or syllables should be experimentally deter- 
mined and objectively recorded to some extent at least, if 
they are worthy of our attention in matters of publicity. 

The experiment to be recorded is a mere outline of what is 
experimentally feasible. Only 15 subjects were used, and for 
purposes of simplifying data, and to study special types of 
articulation, we confined the experiment to certain letters only, 
neglecting the vowel “i” for example. We used only non- 
sense syllables,—so far as possible those which had no sound 
or sight similarity to real situations. This type of stimulus 
has two advantages: it minimizes the significance of associa- 
tions and early affective conditioned reflexes arising there- 
from; and it also tends to approach a practical condition in 
advertising. Many trade-names are nonsense syllables or 
combinations of such syllables so far as the public is concerned. 
It is one way of making a symbol vivid and distinct (e. g. 
rit, tirro, hag, zet, loju, kef, pax, raz, vital, zim). 

Our method was to make up a list of syllables, 50 in all, 
and present them in pairs to the subjects. Each syllable 
occurred twice, each time with a different mate, once the first 
of a pair and the other time the last. This presumed to 
eliminate differences due to position, and to compensate some- 
what for the influences of specific syllables or sounds on 
others. A subject was asked to force a distinction between 
the two syllables shown, and to grade each differently on a 
scale of 3 points. That is, it was the paired associates method, 
allowing at the same time for some grading, so that if the 
two syllables were markedly different in affective quality, one 
might be graded “3” and the other “1.” Each subject was 
asked to think of “3” as significantly pleasing and well liked ; 
of “2” as above average in affective tone, at least not dis- 





1Cf. Bliss Perry, “ A Study of Poetry,” 1920. 
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pleasing ; and “1” as below average, or positively distastetul. 
A more extensive scale of measurement, say a five place 
system, might have been used to advantage, though it would 
require a longer time for decisions which might in turn have 
allowed for the development of further associations in the 
case of syllables used. After each judgment had been made, 
the subject was asked not only to indicate the number to be 
assigned to the right and left syllable, but also indicate after 
a number a plus sign if there was a pleasant association or 
meaning attached, and a minus sign if there was an un- 
pleasant association. There were only three out of the fifty 
syllables used which seemed to call up associations for the 
majority of the group; although of course this method does 
not account for unconscious affective influences arising by the 
conditioned reflex route. 

The second part of the experiment came ten days later, 
without the subjects’ knowledge that they would be expected 
to remember the syllables they had seen. Each of the syllables 
had matched with it a confusing syllable, only one of the 
three original letters being altered. The one hundred syllables 
were presented to the subjects in random order, and the sub- 
jects were asked to indicate those of the group which they 
had seen before. They made a distinction also between those 
they were sure of and those they thought they had seen, but 
were less sure about. 

The results are suggestive. Out of the 50 original syllables 
shown, the following—in the order from high to lower affec- 
tive value—turned out to be distinctly pleasing to the subjects: 

“Nep, lon, dur, paz, des, zef, tor, dos, reg, ros, ber, tur, 
tul, ket, tov,” 

Those syllables most unfavorably considered for the 15 sub- 
jects, in order from medium to the least pleasurable forms 
were: 

“ Bod, seg, sev, kuv, tas, kaz, ruv, sab, rav, geb, vab, vud, 
ged, gur, gak.” 

On account of the large number of “ neutral ” syllables, of 
little difference in affective value, in between these lists, there 
is unquestionably more difference between “ Nep” and “ tov,” 
or between “bod” and “gak,” than between “tov” and 
“bod.” The syllables in the medium affective range, in the 
order of more pleasing to less pleasing were: 

“ Dav, lof, teg, keb, bep, gof, sef, teb, lug, ret, mul, pev, 
sud, zud, reb, dap, kop, bov, sof, kut,” 

An inspection of those syllables accompanied by associa- 
tions leads one to feel that the matter of favorable or unfav- 
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orable associations was not very significant in fixing the order 
in the series of any syllable. For example, the form “ dur” 
had 4 positive (favorable) associations out of the 30 chances 
(2 for each subject), and 2 negative associations. The form 
“lug” had 2 positive and 6 negative associations attached; 
while “sud” with the largest number of associations noted 
had 10 negative and 6 positive marks. ‘The form “ gof” 
which stands at about the middle of the series had 8 positive 
and no negative associations recorded against it, while the 
form “ gak” which is far below all the others in its affective 
quality only had 7 negative associations noted. All the other 
forms had 5 or fewer positive and negative marks, out of a 
possible 30, indicating associations. 

The following preliminary conclusions seem to me justifi- 
able: Forms involving the letters “t,”’ “d,” “1” and “r” 
tend to be pleasurable. There is not enough evidence for 
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n” and many other forms. Forms involving “g” and “ v, 
and to a less extent “k” and “ss,” tend to be unpleasurable. 
It is interesting that of the two labial occiusives, “p” and 
“b,” the latter seems to stand less favorably, perhaps because 
it involves more palate movement. *©~ 

The beginnings of such simple forms seem to be more indi- 
cative of the probable affective tone than ihe endings, or the 
vowel; i. e. there is more regularity between the presence of 
certain letters and the affective quality of the syllables as a 
whole, when we consider first letters. 

A very important question that arises is :—do pleasing letter 
and syllable forms when put in combination always remain 
pleasing, and will the insertion of an unpleasant form neces- 
sarily tend to lessen the affective tone of a combination? Only 
a few random cases of the combinations of such syllables were 
tried; i. e. the presenting of a five or six letter form made 
up from these syllables. On the basis of the results obtained 
from a few subjects, it appears that the affective quality of 
a combination of two syllables can in general be predicted 
from the affective character of the components, but that the 
first of the two combinations is far more potent in determining 
the type of affective tone than the second. The forms “ torav ” 
and “desev,” for example, ranked fairly high, though the 


“ 


final syllables of each were considered quite unfavorable,—as 
noted in the above lists. It is also possible that many com- 
binations, whose components are separately disagreeable, may 
be of average affective value at least. This seemed to be true 
of the form 
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ravab.” 


\ 











4 
/ 
: 
: 
j 


202 JONES 


The relationship between letters and syllables and memory 
value is not as close as in the case of the affective quality. As 
already indicated, the memory value of a syllable was gauged 
by the number of persons recognizing it, among other syllables 
specially intended to confuse, after a ten day interval of time. 
Syllables of highest memory value, beginning with the most 
easily recognized stand in this order: 

“ Bep, sud, sof, tov, reb, tul, ruv, sev, tas, bov, des, kuv, pev, 
paz, sef,” 

Those standing lowest in memory value in the order from 
medium to the lowest are: 

“Gof, lon, kut, dos, bod, teg, ros, mul, ket, kaz, keb, geb, 
ged, dav, vab.” 

Associations are of slightly more value in the case of recog- 
nition than in determining affective tone, the 6 women appar- 
ently benefitting more than the 9 men subjects in this respect. 
Seven out of 15 syllables standing highest in memory value 
were designated as having associations by one or more sub- 
jects, whereas only 4 of the 15 syllables lowest in memory 
value had associations. 

It seems probable that syllables starting with “s” and “t 
are favored as to memory value. This seems also true of 
“r” and “p” sounds. The endings in “v” are apparently 
specially high in memory value. It is interesting that all of 
these letters are distinctly ‘ front of mouth’ forms, involving 
lips or tongue tip, though “v” is also a palate form. The 
letters “g” and “k,” standing low in memory value, are dis- 
tinctly ‘back of mouth’ forms. It is apparently not the 
amount of articulation in a form which determines its mem- 
ory value as some authors have suggested. 

There is perhaps also some connection between vowels and 
memory value. Of the seven syllables most readily recog- 
nized only two have “a” or “e” as vowels, whereas the 
lowest seven syllables all have “a” or “e” centers. How- 
ever, except for the extremes, this relationship does not hold 
and it is therefore of doubtful importance. 

Apparently there is little or no relationship between pleasure 
value and memory value in simple syllables. The correlation 
between these two factors measured objectively, as described 
above for the 50 syllables, is barely. positive—less than .1. 
Those syllables with the highest memory value are apt to be 
mediocre or neutral in affective quality. Several syllables 
which were easily confused in later recognition were quite 


agreeable, e. g. “lon, dos, ros, ket, and dav.” However there 
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is one serious source of error in considering the memory value 
of single syllables which should not be minimized; i. e. the 
difficulty of testing rigidly and comparably each of * the 50 
syllables used. It was obviously impossible to incorporate 
into the recognition test, of 100 syllables, 50 confusion 
syllables each equally different from one of the older 50 
syllables which had been judged for affective value. 

One additional point of interest was studied,—the rela- 
tionship between the pleasure rating, and also the memory 
value, of the initial 50 syllables and the commonness of these 
syllables in the ordinary word forms of every day life. An 
abridged dictionary was referred to, and a notation made of 
the relative frequency of each of the syllables standing high 
and those standing low in affective value, and in memory 

value. There was no relation evident between the prevalence 
of syllables in our every day language and the memory value 
of these syllables. Of the ten nonsense syllables lowest in 
memory vaue, 4 were new or unfamiliar and 4 others were 
only slightly used; of the 10 syllables highest in memory 
value 3 were new and 4 only slightly used. 

Probably more of a relation exists between the affective 
character of syllables and their frequency in language forms. 
Especially is it true that totally unfamiliar syllables tend to¥ 
be disagreeable. Six out of 8 of the syllables lowest in memory 

value were practically unused syllables. It is quite possible 
that the fact that these syllables are unfamiliar does not cause 
their unpleasantness ; on the contrary language usage may have 
“selected” for its permanent forms those syllables which 
were the most pleasing. Obviously, however, any disagree- 
able form repeated often enough will lose much of its un- 
pleasantness. 

‘ In summary, it is evident that the affective and the memory 
value of different letter and syllable forms can be experi- 
mentally studied to the advantage of Applied Psychology. The 
results obtained from a preliminary experiment are in line 
with opinions occasionally expressed. Forms involving the 
liquids “1” and “r” tend to be pleasing. The dentals “t” 
and “d” are distinctly preferable to the harsher occlusives 


“g” and “k,” and to the fricatives “v” and “s.” Initial 


” 


letters, or syllables, of a combination are specially significant 
in determining the affective status of that combination. Forms 
which necessitate lip and ‘front of mouth’ movements have 
the advantage in memory value. There is no simple relation 
between tke pleasure value and the memory value of different 





4 
; 


204 JONES 


forms; but on the average the ‘front of mouth’ forms seem 
to have the advantage in both respects. The relative famili- 
arity which we have with syllables in a language does not 
seem to affect their memory value, nor their affective tone to 
a significant extent; however, the most disagreeable syllables 
are apt to be unfamiliar. 

Such a preliminary experiment can but point to future lines 
of investigation, detailed and objective. Some of the ques- 
tions for possible research are as follows: 

Are the above results confirmed with more extensive experi- 
mentation with many more combinations of letters and 
syllables, and when different methods are used to measure the 
affective and memory values? 

What is the influence of repetition on the affective tone of 
syllables? May certain relatively neutral forms become quite 
pleasing ? 

Will the affective tone of a syllable, and possibly its memory 
value, differ markedly when we consider different articles for 
which it may be a symbol (e. g. automobiles vs. food stuffs) ? 

Are there emotional experiences other than simple affections 
which should be taken account of in favoring a letter or 
syllable form in considering publicity? i. e. may certain forms 
be more “ solid,” or more “ elegant,” than others regardless of 
their pleasure value? 
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SUBSEQUENT HISTORY OF E——-; FIVE YEARS 
AFTER THE INITIAL REPORT 


3y Leta S. HoLtinGwortH, CHARLOTTE G. GarrIsoNn, and AGNES Burk! 
Teachers College, Columbia University 


In THE JOURNAL oF AppLieD Psycuotocy for 1917 a report 
was made of the mental status and educational achievement 


of a boy, E——, at that time 8 years old.1_ Reference to this 
initial report will show that at the time of first mental exam- 
ination E—— showed an IQ of at least? 187 (Stanford-Binet), 


and that he was then doing superior work in the 6th grade 
of the Horace Mann School. In this school the median IQ 
of the pupils is about 116, and the median age for pupils in 
the 6th grade there is 11 to 12 years. 

It is the purpose of this paper to give a brief account of 
the development of E—— five years after the initial report. 
Mental and physical measurements have again been made 
and scholastic achievements have been ascertained, as follows. 
Additional facts, also, concerning E——’s genealogy are 
herewith presented. It is a question of good taste, and per- 
haps to some extent of ethics, as to whether an investigator 
may give the name of a living subject of scientific research. 
Since E has already become well known in the community 
where he resides, and since the facts as now set forth are un- 
doubtedly sufficient to identify him, there seems no real point 
in omitting his name from this report. Permission has there- 
fore been sought and granted for the use of E ’s name, 
which is Edward Rochie Hardy, Jr. 

Additional data on Genealogy. In the initial report data 
were given concerning the parents of Edward. Fairly com- 
plete genealogical records of maternal ancestry are avail- 











Garrison, Burke, and Hollingworth, The Psychology of a Pro- 
digious Child. Journal of Applied Psychology, 1917. 
2“ At least” because E on first examination passed some tests 





at the highest level afforded on the Stanford-Binet. Had still more 
difficult tasks been provided, he might have succeeded with some of 
them, though in the opinion of the examiner he probably would not 
have done so. 
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able.**°. Five persons bearing the surname of the mother 
settled in New England before 1650. These were probably 
all related to each other. The individual who was Edward’s 
direct ancestor first appeared in New England in 1639, and 
settled in Cambridge, Massachusetts. This family attained 
great distinction in the six generations recorded in the New 
England genealogy. A son of the first ancestor in America 
was a royal councillor, and the greatest merchant of his day 
in New England. A grandson was royal governor of Massa- 
chusetts, and later of New Jersey. He was also a patron 
of learning. A great-grandson was chief justice and lieutenant 
governor of Nova Scotia. A great-great-grandson was a royal 
councillor of Nova Scotia, some of whose children settied 
again in England, of whom a son was a distinguished naval 
officer, atttaining the rank of rear-admiral in the British navy. 

These genealogical records, and other records of New Eng- 
land families which intermarried with this family, have not 
been brought up to date. The maternal surname appears first 
about 1176 in the records of England, and was apparently 
Norman-French in origin. 

Mental Measurements. In the spring of 1920 Edward took 
the Thorndike Mental Tests for Freshmen, for entrance to 
Columbia College. An official letter from the Director of 
Admissions at Columbia College states that, “In the Fresh- 
men Tests he (Edward) was number two, out of 483 entering 
Columbia College.” Edward was then 12 years 0 months 
old, the median age of his competitors being about 18 years. 

On September 29, 1921, Edward was examined mentally 
by Dr. Hollingworth, by means of Army Alpha, Form 5 and 
Form 6, for purposes of recording his mental development. 
On Alpha, Form 5, he made a score of 194 points, finishing 
several of the tests before the time limit, without error. On 
Form 6 of Alpha, which was taken subsequently on the same 
afternoon, his score was 201 points; and here too some of 
the tests were finished in about two-thirds of the time allowed, 
without error. As the method of scoring Alpha does not 
provide for a time bonus, this cannot be taken into account 
in the formal score. 





® Bartlett, J. G., The Belcher Families in New England. N. E. Hist. 
and Gen. Reg., vol. 60, pp. 125-136, 243-256, 358-364. 

* Belcher, Jonathan, The Belcher Papers. Coll. of the Mass. Hist. 
Soc., vol. 6 and vol. 7. 

5 Appleton’s Cyclopedzia of American Biography, vol. I, 1887. (Jon- 
athan Belcher.) 
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Physical Measurements. On September 29, 1921, measure- 
ments were taken of height, sitting height, and weight, as 
follows: ht.==64.2 inches, sitting ht==31.7 inches, wt==166 
lbs., (with summer clothing). Health remains most excellent 
This child has never had a serious illness of any kind. 

Scholastic Achievement. In the spring of 1917, Edward 
finished the 6th, 7th, 8th and 9th grade work at the Horace 
Mann School. He was then just 9 years old. Thereafter 
he attended The Friend’s Seminary, and graduated from the 
high school there in the spring of 1920, with an excellent 
record, and excess credits. He was then 11 years 10 months 
old. 

By this time, also, Edward had passed the comprehensive 
examinations of the College Entrance Board for Harvard 
College, one of the present writers having seen the official 
communication from Harvard authorities, making statement 
to this effect. Edward’s maternal ancestors had attended Har- 
vard (one of them having graduated from there at the age 
of 18 years, according to the records), but Edward expressed 
a desire to attend Columbia, and received permission to take 
the mental tests with the applicants of 1920. He was ad- 
mitted to Columbia College with the freshmen of 1920, with 
14 points of advance credit toward a B.A. degree. 

The present writers have at hand an official statement of 
this child’s academic status on June 1, 1921, at the age of 
not quite 13 years. He had then 46 points of academic credit 
toward a B.A. degree in Columbia College. During his fresh- 
man year he made 32 points, maintaining consistently a grade 
of B, except in two subjects. In physical education his rating 
was C, and in contemporary civilization he made A the first 
semester. (It is worth saying that in contemporary civilization 
the final examination® had been objectively standardized by 
Mr. Ben D. Wood, expert in educational measurement, and 
did not depend on the estimate of instructors). These marks 
have the usual significance: A excellent, B= good, C= 
fair, D= poor, F = failed. 

Edward attended the Summer Session of 1921 at Columbia, 
making five credit points, all A grade, and is at this date, 
September, 1921, a sophomore in Columbia, with many points 
of credit in advance of minimum sophomore status. In addi- 








6“ Hardy’s score on the objective examinations of both terms was 
in the highest tenth of the highest percentile.” B.D. W. (The in- 
structor’s estimate, as well as the result of the objective examination, 
enters into the term grades recorded, and indicated above, in this 
course; so that the final grade in the second semester is but B.) 
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tion to having passed the comprehensive examinations for 
college entrance, he has passed the examinations in trigo- 
nometry, solid geometry, chemistry and physics. He is 13 
years 3 months of age. 

Extra-Curricular Activities. Edward was, of course, a con- 
spicuous freshman, because of his extreme youth, and he was 
“hazed ” by the sophomores for refusing to wear a prescribed 
necktie. One of the New York newspapers commented on his 
conduct under hazing as follows: 

“He has demonstrated that he is, nevertheless, a regular 
fellow. He did it first by bringing about a conflict, in which 
he himself was the much-buffeted prize of battle, and then 
by glorying in his bruises instead of making them the basis 
of a grievance. 

“He is a good sport, as well as a good scholar, and being 
both he ought to go far.” 

Edward also participated in the class play, given in 1921, 
humorously consenting to impersonate himself. 

Manual work has no more charm for Edward than it had 
when he was 8 years old. That he can work with his hands 
and with materials when motivated is suggested by an incident 
connected with the liberty bond drive. His teacher relates 
that Edward wanted to pay for his own bond; so he made 
jelly, working at it till very good jelly was made, and sold 
it for the purpose specified. 

In recent years there has developed a keen interest in 
detective stories. 

Comments of Teachers. Comments frorn Edward’s teach- 
ers during the last five years are indicative of their esti- 
mates: “ The regular course of study has been so easy that 
he has, in several subjects, ngtably English and History, ac- 
complished a great amount of voluntary work outside the 
course.” “An excellent mixer with older students.” “ His 
weekly visits have been a pleasure and anticipation, and his 
ability to understand without English the spoken Latin and 
the authors as I have read them aloud to him, has been ex- 
traordinary.” “ Has done very remarkable work in science, 
particularly in theory.” “I predict for him a great scholastic 
record in college.” ‘I consider it a privilege to have had 
something to do with teaching him.” “ Possesses a power in 

atin that few persons ten years his senior can boast.” “ Has 
shown devotion to the best interests of the school.” 

Summary. In the five years which have elapsed since Ed- 
ward was first tested mentally, he has shown no tendency 
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to become mediocre. His gifts have not grown less. He 
maintains his superior status in mental tests. As for achieve- 
ment, he has passed during this interval from the 6th grade 
of the elementary school, half through the second year of 
college. Average children, the country over, born when this 
child was born, and measuring 100 IQ when he measured 
187 IQ, are now in the 7th grade of the elementary schools. 
Edward’s development in the traits and achievements meas- 
ured may be summarized as follows: 

Born June 17, 1908. (According to official birth certificate, as seen 
by Dr. Hollingworth, and according to statement of mother, as received 
by present writers.) 


November 4, 1916. E. *Norm 
Serre | ee eer 49.5 inches 
WN a hicinins eae ES ere 54.2 lbs 
eee 8 Sr reer IQ 100 

status (Stanford-Binet) (Stanford-Binet ) 
as Secs bo bia moneee errr reer er *3rd grade 
status elementary school elementary school 


Serer ee 58.2 inches 
Wc ceeds cr ceee eed EE Se eee 89.5 lbs. 
(light clothing) 

Intellectual... .........ss<. ED ire ciouisas et acwaen- 47 points 
status (Alpha, Form 5) ( Alpha) 
OS eee rr *8th grade 

status college elementary school 


It is an interesting theoretical question as to how far human 
intelligence may vary from the norm, in the direction of 
superiority. The case of this child has been placed on record 
largely because it seems probable that such cases represent 
very nearly the extreme possible limit of variation in the 
human species, as it now exists. At 8 years of age his IQ 
stood at +11 P.E., (1 P.E. being+8 IQ, according to Ter- 
man’s distribution). The probabilities are usually regarded 
as slight that cases beyond 5 P.E. will occur. Perhaps the 
range in human intellect is much greater than probabilities 
would lead us to guess. 

Edward still wishes to be a minister, and go abroad as a 
missionary. To this end he interests himself especially in 
history, the languages, and anthropology. 





* Note: Norms for height, weight and school status are taken from 
Bird T. Baldwin, as established on children from Horace Mann School, 
Francis Parker School, and the Elementary and High Schools of The 
University of Chicago, where the children are above average (for the 
population at large) in intelligence. 
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Since the initial report of this child’s qualities, readers have 
occasionally asked with what meaning the word “ prodigious ” 
was used in reference to him. It was used in the dictionary 
sense of “ wonderful,” “ extraordinary.” 

In these reports there is no intention to approve or to dis- 
approve the educational regimen pursued. Who knows what 
should be the educational treatment of a child standing at 
+11 P.E. in intellect? The sole intention is to record facts 
concerning the identification and development of a deviation 
so extreme that the chances are theoretically almost nil that 
it would occur at all. 
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THE SIGNIFICANCE OF UNWILLINGNESS TO BE 
TESTED 


By F. B. Knicut, State University of Iowa 


As a pedagogical device in introducing the ‘ mental test’ 
to the teachers of a county in lowa, | had the teachers take 
Thorndike’s College Entrance Examination, Part I. At the 
time of taking the test the teachers were given the choice of 
signing their names or not as each saw fit. They were assured 
that no administrative use of any kind would be made of the 
scores. The superintendent would not be informed of the 
relative success of any teacher. However, those teachers who 
signed their names to their tests would be told what scores 
they made and how their scores compared with scores made 
by college students, Stenographers, soldiers, and the like. 

There were 65 teachers in the group, 39 signed their names, 
26 did not. The distributions of these two groups are as 
follows : 

Frequency of Frequency of 
Score scores on signed papers scores on unsigned papers 
35 1 
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The average score of the signed papers was 100. 
The average score of the unsigned papers was 75. Only 
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four of the 26 unsigned papers equalled or exceeded the aver- 
age of the signed group. Only 5 of the 39 signed papers 
were as low or lower than the average of the unsigned tests. 
We have very definite contrast in performance between these 
two groups. Chance variations even with as small a group 
as this would be an unlikely explanation of the difference of 
the two distributions. 

Although we have too few cases upon which to base a cer- 
tain conclusion it is evident that unwillingness to sign one’s 
name to a mental test may be an unconscious confession of 
probable inability to do well. Willingness or even desire to 
know how you compare with others is associated with rela- 
tively good scores. This explanation seems to me more rea- 
sonable than that unwillingness meant becoming modesty or 
willingness inordinate forwardness. Should not one’s attitude 
toward a test be of definite diagnostic aid? 
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NOTES AND COMMENTS 


Announcement of the Psychological Corporation is made in a recent 
number of Science. 

In answer to the letter addressed last ‘year to psychologists in 
regard to the plans for the corporation, 161 replies were received. 
These were all favorable with the exception of two letters from 
psychiatrists. There was general expression of appreciation of the 
importance of an organization that will maintain adequate standards 
in the applications of psychology and will use any profits there may 
be for psychological research. 

The directors have authorized the establishment of branches where- 
ever a director or three members of the American Psychological Asso- 
ciation are ready to take an active interest in the work. We shall be 
glad to receive word from those to whom this memorandum is ad- 
dressed in regard to the formation of a branch in their neighborhood. 

The stock of the corporation is to be held by psychologists active 
in its work. The directors have authorized the sale of from one to 
three shares to competent psychologists. The stock will be sold for 
$10 a share and may be taken over by The Psychological Association 
at $100. The principal capital of the corporation is the ability and 
interest of those who take part in its work, and a psychologist who 
subscribes $10 for a share is expected to give time worth $90 before 
disposing of the stock, though only as a general understandng, the 
decision regarding which is left entirely to the holder of the stock. 
Those to whom the memorandum is addressed who want one share 
of the stock may send a check for $10 to the treasurer of the Psycho- 
logical Corporation, and the stock will be forwarded. If they want 
one or two further shares application should be made and the shares 
will be allotted if available. 

J. McKeen Cartre.t, President. 


The Psychological Corporation, 
2617 Grand Central Terminal, 
New York City. 
February 20, 1922. 


Information Course on Science of Management and Applied Psy- 
chology. 

From August 21 to August 26, 1922, an Information Course on the 
Science of Management and Applied Psychology will take place at the 
“ Orga-Institut”"—an Institution devoted to the Science of Manage- 
ment and Applied Psychology in Berlin. 

This Course will include lectures in the lecture-hall of the Institu- 
tion and practical exercises in the laboratory as well as inspection 
trips with descriptive lectures. 

It is planned to visit the following Institutions: the Psychotechnical 
Laboratory of the United Berlin Street Car Companies (employment 
psychology and vestibule school), the Psychotechnical Laboratory of 
the Head Postmastery, the Laboratory for Industrial Psychology at 
the Technical High-School-Charlottenburg, the Psychotechnical Lab- 
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oratory of the Berlin Office of Labor, a number of Psychological 
laboratories of large Berlin industrial concerns, the Institute of Ap- 
plied Psychology directed by Geheimen Sanitatsrat Dr. Moll (specially 
for pathological cases) as well as some other places if the time set 
suffices. 


The languages spoken at the Institute are German, English and 
French. 

Only a limited number of students will be admitted, in order to 
give every member of the course the utmost possible consideration 
and to enable him to get full insight into the problems to be dealt with. 

As all applications will be considered according to the date of re- 


ceipt, it is advisable to give notice of attendance at the earliest pos- 
sible moment. 


The fees are for: 
Dutch, 75 Gulden 
Swiss, 150 Francs 
Danes, 150 Crowns 
Swedes, 150 Crowns 
Norwegians, 125 Crowns 
Englishmen, 5 £ 
Americans, 25 Dollars 
Italians, 400 Lire 
Spain, 150 Pesetas 


For other Nations fees will be in accordance with these rates. 
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BOOK REVIEWS 


Harry Dexter Kitson. The Mind of the Buyer, A Psychology of 
Selling. New York, The Macmillan Company, 1921. Pp. x+211. 
Price, $1.50. 

Not to treat of the technique of selling, but of certain psychological 
principles related to the mind of the buyer, such is the purpose of this 
book. Most of the illustrative material, so far as it bears directly on 
selling, has to do with advertising. The outline treats of the usual 
stages in the sale, with good analysis of the important topics: atten- 
tion, interest, desire, confidence, decision and action. In addition, a 
final chapter on the stage of satisfaction brings in an emphasis which 
is one of the best things in the volume. The salesman, advertiser, and 
sales correspondent, for whom the book is written, will find it quite 
readable and brief. The author has a facility for catchy phrases which 
stands him in good stead. Occasionally the psychological terminology 
clings to the pages in spite of the author’s attempt to escape it. Dwell- 
ing on images, for example, seems unnecessary. 

The psychologist will be interested in Kitson’s excellent use of the 
historical method in studying advertising in order to discover the drift 
of progress through the decades. That advertising has made progress 
by its hit or miss methods is well shown by his studies of the decrease 
in the use of superlatives, decrease in the use of all capitals for head- 
lines, and analysis of the kind of information used in advertisements. 
One regrets that more concrete examples of his principles from the 
field of personal selling were not available to the author. The illus- 
tration of the method used by a canvasser to sell a cooking product 
to a housewife is good, but is somewhat lonesome. Since the book 
meets a popular demand, often uses figurative language admirably for 
this purpose, and is generally sound in its principles, the reviewer may 
perhaps be most useful in wishing that somebody would build a dif- 
ferent kind of book based on a psychological analysis of a collection 
of methods used by successful salesmen and illustrated with details of 
the different methods. J. B. MINER. 


Paton. Hwman Behavior, in Its Relation to the Study of Educational, 
Social and Ethical Problems. New York, Charles Scribner’s Sons, 
1921, pp. 465. Price, $6.00. 


To the student of applied psychology no work would be more wel- 
come than one on human behavior which should open a broad highway 
into that complex and difficult subject. Such a highway is, however, 
too much to expect at the present time. The practical control of 
human behavior has been so long a major task of so large a part of 
mankind that all the easy portions of the field have long been familiar, 
while its deeper penetration has had to wait for the advance of 
knowledge in other fields, especially in those of biology and physiology, 
if indeed the means are even yet available for any great advance of a 
rigidly scientific sort. For this reason it is enough if Dr. Paton blazes 
us a trail; in fact, he himself refers to his work as an “ introduction,” 
and in his exposition finds himself often brought to a stand by the 
absence of necessary information not yet available. 
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The main thesis of the book is that the key to the complexities of 
adult human behavior is to be found in the simpler sorts of behavior 
appearing in the lower, and even lowest, animals, in the earliest stages 
of human life (including the prenatal) and in the disordered processes 
of mental and nervous disease. The study of behavior involves the 
study of the responses of the total organism to the total environment. 
One must go not only back of the brain and spinal cord to the autono- 
mic nervous system, but back of that to the endocrine glands, the 
muscles, the vegetative system and even to the most rudimentary 
adaptations of living matter; and above all one must not separate the 
mental from the physical. 

The first of Dr. Paton’s fourteen chapters. traces the historical de- 
velopment of this biological-physiological point of view. The next 
eight discuss behavior (and personality, its unified expression) in its 
dependence upon the bodily mechanisms of adjustment and control, 
upon “trends” in activity and upon habit-formation. The next two 
deal with the typical problems of degeneration in old age and of the 
imperfect organization of personality appearing in conflicts and dis- 
sociations; chapter twelve with the method to be used in studying 
personality; chapter thirteen with education; and chapter fourteen 
with the more general sociological and international results that may 
be expected to follow from the study of the causes of human action. 

To the general reader the chapter on Education will perhaps prove 
most interesting and can hardly fail to win the hearty approval of any 
who know what current education in the home and schools actually 
is and what it might be, but those who have occasion to use the book 
as a whole will regret that Dr. Paton has not employed more uni- 
formly the direct and luminous style of which he shows himself occa- 
sionally to be the master and that he has chosen to present so large a 
part of his material in so abstract a manner. A few detailed cases 
would have been both more interesting and more illuminating. 

EDMUND C. SANFORD. 


Developing Mental Power. By Stratron, Georce M. Boston, Hough- 
ton Mifflin, 1922. 77p. $.80. 


Appearing as one of the Riverside Educational Monographs this 
book is patently a book on Educational Psychology for teachers, 
treating of the questions, what is mind, and how shall it be trained? 

The opposing views of extreme faculty psychology and extreme 
particularization of function are presented, though naturally the sup- 
porting arguments of each could not be very largely developed within 
the compass of seventy-seven small pages. The educational implica- 
tions of each view are treated by means of reviews of the discussions 
centering around transfer of training. The conclusion is reached that 
certain habits can be generated through school discipline, habits of 
giving “controlled attention to the task in hand; energetic attack upon 
it; accuracy in interpreting, remembering and reporting what is seen 
or read or heard; the power to distinguish important and unimportant. 
These are part of intellectual training; these and other things take the 
place of the few faculties of the older belief.” Some of the other 
things are, emotional and volitional control which the author gives an 
important place in the educational process, even going so far as to 
suggest exercises in “will training’’ which the teacher can give the 
child: “Parents and teachers might well invent and assign things to 
be done, rewarding in themselves, and chosen, perhaps, from cooking, 
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drawing, modeling, painting, acting, reading, or any other of a hundred 
things—but now used in order to make habitual the right ways of 
purpose, applicable in any work. These right ways might here be set 
down, with another purpose than was guiding us earlier in this section, 
as: (a) suitable forethought; (b) speed and energy of attack, once 
the decision is made; (c) perseverance in what is undertaken; (d) 
economy of action, elimination of waste effgrt, “form”; (e) excel- 
lence of result in the product; (f) restoration of order when the work 
is done, putting away of tools and materials, clearing and cleaning up. 
Each of these six phases of the process should receive due attention,— 
perhaps one at a time, as Benjamin Franklin practiced the virtues,— 
but recurring and with different degrees of difficulty. There should be 
brief explanation before and after the fact, that the idea of what is 
sought should come with the practice, and should help to make the 
practice itself more fruitful and ready to reappear spontaneously in 
new places. And whatever is approved elsewhere as a means to in- 
terest and progress might be used here; if “ marks,” rewards, praise, 
or rivalries are good to spur on in numbering or writing or any other 
study this present learning to will aright is as worthy of their incentive. 

As an exercise in suitable fore thought, the following might serve 
as a door to something better. There is, let us say, but ten minutes 
left, and the child must choose between cutting some design in paper 
and making candy; and the choice is then appraised, with explanation, 
according as the child has stopped to think, to look ahead, before de- 
ciding. Or, again, having at hand only some modeling wax, a pair of 
scissors and some very narrow strips of thin colored paper, one must 
decide whether to build a paper house or make the figure of a dog. 
Or still again, the child, without actual materials at hand and with the 
use only of his imagination, must say—with no change of vote per- 
mitted—which line of conduct is suitable, either in cases like those 
just given, or where some one of a thousand other situations is de- 
scribed—where, perhaps, a child has visiting playmates who have come 
walking from afar and up a long steep hill; shall they at first play 
“authors” or play “tag”? 

Other evidences that the author has not been able entirely to avoid 
the extravagances incidental to popular psychologizing are such sen- 
tences as: “Steadiness [of will] not only has ten times the effect of 
violence, it is ten times more readily attained.” 

Nevertheless the monograph presents some considerations that may 
be new and instructive to young teachers even in these days when 
nearly everyone who goes to normal school takes a course in elementary 
or educational psychology, and by its stress upon the need of experi- 
mental verification of educational practices it should perform a ser- 
vice, to teachers, to psychology and to education. H. D. KITSON. 


Psychotechnik und Taylor—System. K. A. Tram. Berlin, 1921, mit 
89 abbildungen, VII, # 139. 


Critics of the Taylor System have pointed out time and again that 
it has overemphasized the mechanical aspects of work at the expense 
of the human aspects. We know that it was certainly not Taylor’s 
intention to do this, but that he was hampered in his approach to the 
problems of human behavior by the lack of an adequate technique. 

Most of his followers were even less well equipped in this respect 
than he, and consequently, the subsequent development of Taylor’s 
system was increasingly lop-sided. We have, as a result, the technique 
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of time-study and task setting which not only gives inadequate con- 
sideration to the human aspect of work but which is full of fallacies 
from a statistical point of view. Notable exceptions to this tendency 
are the Gilbreths and Robert Wolf. The latter called attention to the 
beneficial effects of giving to individuals graphic information about 
the progress of their work. The former attacked the more funda- 
mental and comprehensive problem of developing the best possible 
methods of doing work by studying tasks in terms of the activities of 
the worker by which they must be carried out. Various methods were 
used to accomplish this, the best known being the photographic motion 
study. 

In this country, the fundamental soundness of the ideas underlying 
the Gilbreth technique has been ignored because of its mechanical in- 
tricacies, so that the Gilbreths are today somewhat outside the pale of 
the more orthodox followers of Taylor. On the other hand, on the 
continent, and particularly in Germany, where the Taylor system has 
at least as great a vogue as in this country, Frank Gilbreth is regarded 
as its foremost exponent. And consequently, the development of the 
Taylor system there has been, at almost every point, influenced by 
Gilbreth’s emphasis upon the study of human behavior and human 
capacities. The extensive literature on scientific management which 
the Germans have developed is an eloquent testimony of this influence, 
for it is difficult for any one to pick up anything on the subject of 
Taylor without finding reference to Gilbreth or accounts of his work 
either there or in this country. And the extensive work which the 
Germans themselves have done has definitely taken the direction of a 
study of the problems surrounding the performance of industrial tasks. 

This is particularly true of the German psychologist. Whereas in 
this country, psychologists have given little or no consideration to Gil- 
breth’s work, even though his technique is the ideal embodiment of the 
behavioristic point of view, in Germany it has been the stepping stone 
by which they have transplanted their methods directly into industry. 
And consequently, while applied psychologists in this country have 
been dabbling with the fringes of industry, German psychologists have 
stepped directly into its heart. 

With this introduction, the significance of Tramm’s title “ Psycho- 
technik und Taylor System,” as well as the contents of the book, can 
be more readily understood. Tramm is one of a group of industrial 
engineers and psychologists among whom are W. Moede, G. Schlessin- 
ger, A. Wallichs, S. Sinner, Alex. Schackwitz, E. Mickel, C. Piorkow- 
ski, who have been active in this field. Tramm begins with an outline 
of the general factors which apply to all work, such as physical en- 
vironment, work periods, living conditions and habits of life, indi- 
vidual and team work, supervision, and the elements into which all 
these factors are subdivided. He discusses them merely in order to 
indicate their importance, without any attempt to show how they are 
to be controlled. Most of his book is devoted to the discussion of 
experiments and methods calculated to show how particular occupa- 
tions can best be arranged with reference to the worker. For ex- 
ample, a study of about one hundred tram motor-car men under ex- 
perimentally controlled conditions resulted in a modification of the 
handles of the three levers which they were required to use and also 
in a standard method by which these levers could be manipulated to 
the best advantage. Throughout the book, emphasis is placed not only 
on the details or steps by which an occupation can be best performed, 
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bu: upon the construction of the tools and apparatus peculiar to it as 
well. The description of the laboratory devices for measuring the 
performance of individuals brings out vividly the contrast between the 
German approach and that of the followers of Taylor in this country. 
Instead of the lonely stop watch from which time is recorded only by a 
fallible observer, Tramm describes in addition a half dozen other de- 
vices of varying degrees of refinement, devices which record opera- 
tions graphically and which can be adapted t6 a wide range of occu- 
pations. Gilbreth’s motion study technique is also fully discussed, 
though without any explicit reference to its originator. 

The effects of practice are investigated with special reference to the 
process of training workers. And one of the outstanding features of 
the book is the presentation of a pictorial technique for instruction 
purposes. The study of fatigue is probably the most unsatisfactory 
section of the book, a fact which can be readily understood when we 
know the nebulous stage in which the fatigue problem rests today. 

The psychologist will undoubtedly find much to criticize in this book. 
Its experiments are not presented with sufficient facts or data to enable 
him to estimate the value of their results. There is a too ready ac- 
ceptance of some of the traditional and worn out concepts of psychol- 
ogy. Inferences are drawn which are often unwarranted by the facts 
And the academic or superficial point of view is still very noticeable 
Industrial engineers on the other hand may well question the practi- 
cability of the technique and the value of some of the results. Never- 
theless, the book as a whole is an admirable attempt to visualize the 
problems of production, not only in relation to output but in relation 
to human capacities and abilities as well. As such it is an excellent 
critique of Taylorism as it exists in this country and should be of 
considerable suggestive value to psychologists, industrial engineers, and 


others who are interested in industrial practices. HENRY C. LINK. 
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Though the manual for administering this test is still in mimeo- 
graph form the tests are ready for use and the mimeographed 
sheet of directions contains all instructions for the successful use 
of the examination. 

The publishers announce that the Higher Examination will be 
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testing any literate person. 
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